How to run conductor-server as a dependency in a Spring Boot Application?

[ceroy-ak]

What?

To start the conductor server as a dependency from my Spring Boot Application to extend its functionality.

Why?

I am trying to implement a feature where every time there is a new message in the Azure Queue Storage, the workflow is triggered to perform a set of tasks.

How am I trying?

My initial thought process has multiple approaches (Not settled with anyone yet):

1. Get the conductor-server, and Azure dependencies in a spring boot application, supply the right config.properties and then extend the application to implement the custom behaviour.

2. Run the pre-compiled jar with the right configs as -DCONDUCTOR_CONFIG_FILE=$CONFIG_FILE, then have a separate Spring Boot application running with the Conductor SDK to Create Workflow definitions, trigger workflows, and other task definitions. And then have another worker spring boot application that would poll the conductor instance for tasks. (Yes I know we can have the same instance do all that, but I want to keep a separation of concern)

3. Clone the git repo, make changes and extensibilities in a branch and then create a docker image with config being part of a separate volume.

---

I am open to all suggestions on how to implement this requirement as I want to use a battle-tested workflow orchestration tool like Conductor. Think of this like what if there was a non-standard tool to be used as part of a requirement, then how can we extend conductor to implement the same.

[ceroy-ak]

To extend point 1 of how am I trying?

I have added the following dependency in my pom.xml file

<dependency>
	<groupId>com.netflix.conductor</groupId>
	<artifactId>conductor-server</artifactId>
	<version>3.13.7</version>
	<classifier>boot</classifier>
</dependency>

and my application.properties looks like the following

# Servers.
conductor.grpc-server.enabled=false

# Database persistence type.
conductor.db.type=mysql

spring.datasource.url=jdbc:mysql://127.0.0.1:3306/conductor
spring.datasource.username=root
spring.datasource.password=root

# Hikari pool sizes are -1 by default and prevent startup
spring.datasource.hikari.maximum-pool-size=10
spring.datasource.hikari.minimum-idle=2

# Elastic search instance indexing is enabled.
conductor.indexing.enabled=false

# Transport address to elasticsearch
conductor.elasticsearch.url=http://es:9200

# Name of the elasticsearch cluster
conductor.elasticsearch.indexName=conductor

# Load sample kitchen sink workflow
loadSample=false

but I am getting the following error on being executed

java.lang.IllegalStateException: Unable to load cache item
	at org.springframework.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:75) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.core.internal.LoadingCache.get(LoadingCache.java:34) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:126) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:313) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.proxy.Enhancer.createHelper(Enhancer.java:562) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.proxy.Enhancer.createClass(Enhancer.java:407) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.context.annotation.ConfigurationClassEnhancer.createClass(ConfigurationClassEnhancer.java:138) ~[spring-context-6.0.9.jar:6.0.9]
	at org.springframework.context.annotation.ConfigurationClassEnhancer.enhance(ConfigurationClassEnhancer.java:109) ~[spring-context-6.0.9.jar:6.0.9]
	at org.springframework.context.annotation.ConfigurationClassPostProcessor.enhanceConfigurationClasses(ConfigurationClassPostProcessor.java:531) ~[spring-context-6.0.9.jar:6.0.9]
	at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanFactory(ConfigurationClassPostProcessor.java:308) ~[spring-context-6.0.9.jar:6.0.9]
	at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:358) ~[spring-context-6.0.9.jar:6.0.9]
	at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:150) ~[spring-context-6.0.9.jar:6.0.9]
	at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:771) ~[spring-context-6.0.9.jar:6.0.9]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:589) ~[spring-context-6.0.9.jar:6.0.9]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.1.0.jar:3.1.0]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:733) ~[spring-boot-3.1.0.jar:3.1.0]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:435) ~[spring-boot-3.1.0.jar:3.1.0]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:311) ~[spring-boot-3.1.0.jar:3.1.0]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1305) ~[spring-boot-3.1.0.jar:3.1.0]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1294) ~[spring-boot-3.1.0.jar:3.1.0]
	at ai.typeface.poc_conductor.PocConductorApplication.main(PocConductorApplication.java:32) ~[classes/:na]
Caused by: java.lang.NoClassDefFoundError: redis/clients/jedis/JedisPoolAbstract
	at java.base/java.lang.Class.getDeclaredConstructors0(Native Method) ~[na:na]
	at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3373) ~[na:na]
	at java.base/java.lang.Class.getDeclaredConstructors(Class.java:2555) ~[na:na]
	at org.springframework.cglib.proxy.Enhancer.generateClass(Enhancer.java:655) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.transform.TransformingClassGenerator.generateClass(TransformingClassGenerator.java:35) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:26) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.core.ClassLoaderAwareGeneratorStrategy.generate(ClassLoaderAwareGeneratorStrategy.java:57) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:362) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.proxy.Enhancer.generate(Enhancer.java:575) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData.lambda$new$1(AbstractClassGenerator.java:103) ~[spring-core-6.0.9.jar:6.0.9]
	at org.springframework.cglib.core.internal.LoadingCache.lambda$createEntry$1(LoadingCache.java:52) ~[spring-core-6.0.9.jar:6.0.9]
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
	at org.springframework.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:57) ~[spring-core-6.0.9.jar:6.0.9]
	... 20 common frames omitted
Caused by: java.lang.ClassNotFoundException: redis.clients.jedis.JedisPoolAbstract
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[na:na]
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na]
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
	... 33 common frames omitted

I have verified that my MySQL instance is running with the given credentials

and my main class has the following annotation

@ComponentScan(basePackages = {
		"com.netflix.conductor",
		"io.orkes.conductor"
})