Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alpakka - BQ behaviour #2904

Open
guda249 opened this issue Sep 8, 2022 · 0 comments
Open

Alpakka - BQ behaviour #2904

guda249 opened this issue Sep 8, 2022 · 0 comments

Comments

@guda249
Copy link

guda249 commented Sep 8, 2022

This is w.r.t to Big Query alpakka library I have following questions.

  1. There are no exception logged or any information is logged , when the BQ servers are not reachable, neither the Application shuts down.
  2. When the Google Credentials json are not passed, the application just logs the info and remains in the running state.

BigQuery.insertAll API accepts case classes for marshalling.

  1. Given the limitation of case class fields to 22. how to pass any data which has higher number of columns > 22 .
  2. Currently there is no provision to set the values for skipInvalidRows and ignoreUnknownValues, the values are set to None by default.

Suggestion needed on the following,

  1. Is it the expected behaviour w.r.t point 1 & 2 ? The ideal behaviour would be the application should shut down, as the upstream servers are not reachable, or retry the communication in exponential manner (in case if persistent connection are maintained, which is not the current case). In the worst case, there should be some information logged at error/info level. Or am i missing any configs which can give the expected behaviour.
  2. Any workaround for the Point 3.
  3. Are there are config's which can be used to pass these values mentioned in Point 4.

Version : 3.0.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants