Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trying to fix integrity errors before migrating tables #12279

Merged
merged 2 commits into from
Jun 18, 2024

Conversation

jredrejo
Copy link
Member

Summary

Before updating kolibri database on a Django upgrade, check integrity of the tables and fix automatically errors that can be safely fixed

References

Closes: #12138

Reviewer guidance

Using a broken db, check that it's fixed.
Using a good db, check no problems are added

Testing checklist

  • Contributor has fully tested the PR manually
  • If there are any front-end changes, before/after screenshots are included
  • Critical user journeys are covered by Gherkin stories
  • Critical and brittle code paths are covered by unit tests

PR process

  • PR has the correct target branch and milestone
  • PR has 'needs review' or 'work-in-progress' label
  • If PR is ready for review, a reviewer has been added. (Don't use 'Assignees')
  • If this is an important user-facing change, PR or related issue has a 'changelog' label
  • If this includes an internal dependency change, a link to the diff is provided

Reviewer checklist

  • Automated test coverage is satisfactory
  • PR is fully functional
  • PR has been tested for accessibility regressions
  • External dependency files were updated if necessary (yarn and pip)
  • Documentation is updated
  • Contributor is in AUTHORS.md

@jredrejo jredrejo added the TODO: needs review Waiting for review label Jun 13, 2024
@jredrejo jredrejo requested a review from rtibbles June 13, 2024 17:20
@github-actions github-actions bot added DEV: backend Python, databases, networking, filesystem... SIZE: small labels Jun 13, 2024
Copy link
Member

@rtibbles rtibbles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One tweak to avoid premature Django initialization, which I also think will help with the testing error.

@@ -185,6 +186,27 @@ def _copy_preseeded_db(db_name, target=None):
)


def sqlite_check_foreign_keys():
db_connection = connections["default"]
Copy link
Member

@rtibbles rtibbles Jun 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is explicitly being done in the method that is before Django setup, so I think using the django DB connection here might not be a good idea. Could you instead generate the connection directly using the Python sqlite library?

We should also iterate over all the SQLite databases listed in https://github.com/learningequality/kolibri/blob/develop/kolibri/deployment/default/sqlite_db_names.py#L14 to be safe.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

argg, got it. Thanks for the advice
Changed as requested

@jredrejo jredrejo requested a review from rtibbles June 17, 2024 10:07
@jredrejo jredrejo self-assigned this Jun 17, 2024
Copy link
Member

@rtibbles rtibbles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks right, I'll test it with the database that @AllanOXDi had with the original issue. Thanks!

Copy link
Member

@rtibbles rtibbles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manual testing with an affected database shows that the fix is applied and logged, and manual inspection of the database afterwards shows that the affected rows have been removed.

@rtibbles rtibbles merged commit 418a2a2 into learningequality:develop Jun 18, 2024
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DEV: backend Python, databases, networking, filesystem... SIZE: small TODO: needs review Waiting for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handle Integrity Error in Django 3.2 Upgrade
2 participants