How to achieve non-blocking parallel task execution in a single worker #16294
Unanswered
shestakov2103
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi! I'm using Payload 3.0 Jobs with the following worker command:
payload jobs:run --concurrency 10 --handle-schedulesIssue: When a task handler contains a long-running asynchronous operation (e.g., a 2-minute generation), the worker stays "busy" and doesn't pick up other pending tasks from the queue until the current one finishes. This happens even with concurrency set in the TaskConfig.
Question: Is the internal worker designed to be strictly sequential within a single process? How can I achieve true parallel, non-blocking execution so that one long task doesn't block the entire queue?
Does concurrency only work across multiple physical worker processes, or can it be achieved within one?
Beta Was this translation helpful? Give feedback.
All reactions