Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nack with delay #102

Open
thedodd opened this issue Sep 29, 2021 · 0 comments
Open

Nack with delay #102

thedodd opened this issue Sep 29, 2021 · 0 comments

Comments

@thedodd
Copy link
Collaborator

thedodd commented Sep 29, 2021

  • Stream & Pipeline consumers should be able to nack an event with a delay.
  • Stream controllers will need to be updated to use a different tracking mechanism for group offsets.
    • Delayed items will need to be recorded as outstanding and recorded on disk (just the offset).
    • If a controller restarts, it will need to be able to detect such delayed events for a group and then spawn them as timer delays until they can be delivered.
  • Pipeline controllers will need to be updated to not block max parallel consumers when there are no consumers for a particular stage, as this would cause all pipeline processing to halt for a partition just because a stage consumer is not active.
    • The metrics system should still alert on such cases.
    • The way we model such a delay for a Pipeline stage will be different, but we should be able to just record this info under a stage specific key for the pipeline.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant