Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Design a system which consumes columnar file notifications from HDFS/cloud storage and generates delta, hudi and iceberg metadata #554

Open
1 of 2 tasks
vinishjail97 opened this issue Oct 4, 2024 · 2 comments
Open
1 of 2 tasks

Design a system which consumes columnar file notifications from HDFS/cloud storage and generates delta, hudi and iceberg metadata #554

vinishjail97 opened this issue Oct 4, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@vinishjail97
Copy link
Contributor

Feature Request / Improvement

Sub-task of main feature request #550

Look at Using cloud notifications queue in the approach mentioned below. It's only a high level approach, with many open ended items. We can do a design doc and finalize the approach through a discussion in dev mailing list.
#550 (comment)

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@vinishjail97 vinishjail97 added the enhancement New feature or request label Oct 4, 2024
@vinishjail97
Copy link
Contributor Author

Tagging people who were interested to contribute @sudharshanraja-db @purnachandergit in the design, let's discuss this in dev mailing list.

@purnachandergit
Copy link

I will be working on this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vinishjail97 @purnachandergit