This repository has been archived by the owner on Dec 13, 2023. It is now read-only.
Replies: 1 comment 1 reply
-
Consider defining a task definition for your HTTP tasks with the retry count and retry strategy as desired. |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
We are using conductor version v3.10.6 with REDIS as queue.
We have 25 replicas of 1 cpu 1gb memory conductor servers running on the cloud.
When I'm trying to run 6000 workflows that contain simple HTTP tasks only ,in parallel
The scheduling time for each http task can reach up to 2 minutes.
Which is way too slow for our needs.
Hence I've read here That I can change the HTTP task def to is async false.
Ive tried it and indeed it made the entire operation run 2 times faster without any problems.
Until I noticed It's not possible to use exponential backoff retry for those http tasks now that they have async false.
I googled around and found this thread talking about this issue here
Which is required for us.
So how/where could I change Netflix conductor code to support exponential backoff for sync http tasks.
Or any other recommendation to make the system faster with support for exponential backoff will be welcome.
We are using elastic version 6.8 with 3 strong replicas.
Thanks!
Beta Was this translation helpful? Give feedback.
All reactions