Handle external_data table deploys mostly the same as for regular tables #6558
+86
−93
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes a bug introduced in #6529 -- this PR added support for tables that are defined with a
metadata.yaml
but noquery.sql
, but didn't check for tables that are built on external data (Google Sheets or CSV)This adds logic to the
deploy_table
function to handle external tables differently. In the process I also did some refactoring of how we deploy external tables. I moved the metadata update logic that was in_deploy_external_data
into anattach_external_data_config
function inpublish_metadata.py
and removed the duplicated code.Is this reasonable? This also cuts one of the passes we're doing through all the sql and metadata files during a deploy, so I think it's a performance benefit. Unless there was a reason that we weren't deploying external tables in the same threads as the rest of the table deploys?
Also, I should add some tests; this bug is kind of my fault for not adding tests in the first place. If it's okay with you, I'd like to do that in a separate PR.
Related Tickets & Documents
┆Issue is synchronized with this Jira Task