Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove unnecessary availability check #3655

Merged

Conversation

rebeccacremona
Copy link
Contributor

Many years ago, we were occasionally finding that, when we uploaded WARCs to S3, they were not immediately available for download, even though the upload was completed. We added a retry to account for this.

Also many years ago, but more recently, Jack looked in to it and reported S3 has since changed, and this inconsistency isn't a problem any more. We decided to keep this check in the codebase anyway, because it was harmless.

Well, now it's causing trouble: we don't have WACZs when new upload-your-own archives are created, because we are still just making WARCs for those. So, the check fails.

Instead of tweaking the check to work properly in both situations, I'm just removing it.

If we find that S3 is, in fact, still inconsistent, then we can add a check back in.

@rebeccacremona rebeccacremona requested a review from a team as a code owner November 15, 2024 16:02
@rebeccacremona rebeccacremona requested review from cmsetzer and bensteinberg and removed request for a team and cmsetzer November 15, 2024 16:02
Copy link

codecov bot commented Nov 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.61%. Comparing base (d4747b8) to head (e631bd3).
Report is 5 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3655      +/-   ##
===========================================
- Coverage    69.61%   69.61%   -0.01%     
===========================================
  Files           54       54              
  Lines         7350     7339      -11     
===========================================
- Hits          5117     5109       -8     
+ Misses        2233     2230       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rebeccacremona rebeccacremona merged commit 06d7f73 into harvard-lil:develop Nov 15, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants