Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: COPY logic fails if schema has updates #53

Open
pnadolny13 opened this issue Jun 11, 2023 · 0 comments
Open

bug: COPY logic fails if schema has updates #53

pnadolny13 opened this issue Jun 11, 2023 · 0 comments

Comments

@pnadolny13
Copy link
Contributor

As described in #52 (comment). The COPY logic has a bug when schema updates are required. The sync fails with an error related to a mismatch in column counts in the copy statement/table definition/internal stage files.

Draft PR up with test case #52 but still needs logic changes to fix them.

@pnadolny13 pnadolny13 removed this from Data Team Jun 12, 2023
pnadolny13 added a commit that referenced this issue Jun 14, 2023
Closes #48

Sqlalchemy handles some of the quoting for reserved words but we also
build sql manually so those dont get handled. Also when doing column
diffing like to decided whether a column exists or not we need to use
uppercase column names because we created them as uppercase with quotes
so case matters.

I found a bug with the COPY logic when the stream has schema updates.
Logged it in #53
and draft PR in #52.
Once that is fixed we can add a schema change to the test records with
no key properties so we assert alters work with reserved words when no
key properties are being used.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

1 participant