Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear off-chain worker activity validation cache #474

Open
wants to merge 17 commits into
base: dev
Choose a base branch
from

Conversation

khssnv
Copy link
Member

@khssnv khssnv commented Nov 12, 2024

Description

The pallet-ddc-verification off-chain worker writes temporary data to the database and the patch implements its data removal when OCW doesn't needed it anymore.

Types of Changes

Please select the branch type you are merging and fill in the relevant template.

  • Hotfix
  • Release
  • Fix or Feature

Fix or Feature

Types of Changes

  • Tech Debt (Code improvements)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Dependency upgrade (A change in substrate or any 3rd party crate version)

Migrations and Hooks

  • This change requires a runtime migration.
  • Modifies on_initialize
  • Modifies on_finalize

Checklist for Fix or Feature

  • Change has been tested locally.
  • Change adds / updates tests if applicable.
  • Changelog doc updated.
  • spec_version has been incremented.
  • network-relayer's events have been updated according to the blockchain events if applicable.
  • All CI checks have been passed successfully

Checklist for Hotfix

  • Change has been deployed to Testnet.
  • Change has been tested in Testnet.
  • Changelog has been updated.
  • Crate version has been updated.
  • spec_version has been incremented.
  • Transaction version has been updated if required.
  • Pull Request to dev has been created.
  • Pull Request to staging has been created.
  • network-relayer's events have been updated according to the blockchain events if applicable.
  • All CI checks have been passed successfully

Checklist for Release

  • Change has been deployed to Devnet.
  • Change has been tested in Devnet.
  • Change has been deployed to Qanet.
  • Change has been tested in Qanet.
  • Change has been deployed to Testnet.
  • Change has been tested in Testnet.
  • Changelog has been updated.
  • Crate version has been updated.
  • Spec version has been updated.
  • Transaction version has been updated if required.
  • All CI checks have been passed successfully

@khssnv khssnv marked this pull request as draft November 12, 2024 11:38
@khssnv khssnv force-pushed the feature/clear-ocw-validation-activities branch from a5f20a9 to 050ab17 Compare November 12, 2024 11:42
@khssnv khssnv force-pushed the feature/clear-ocw-validation-activities branch from 050ab17 to 09e1710 Compare November 12, 2024 12:05
@khssnv khssnv force-pushed the feature/clear-ocw-validation-activities branch from 09e1710 to 5fa3d0d Compare November 12, 2024 12:14
@khssnv khssnv marked this pull request as ready for review November 13, 2024 07:23
@khssnv khssnv force-pushed the feature/clear-ocw-validation-activities branch from 7f7f396 to a13c70b Compare November 18, 2024 07:49
@khssnv khssnv force-pushed the feature/clear-ocw-validation-activities branch from a13c70b to 7f5a974 Compare November 18, 2024 07:52
@yahortsaryk
Copy link
Contributor

yahortsaryk commented Nov 18, 2024

There is an issue in the current implementation. We should remove the cached data (activity aggregates) for an Era only after this payout process for this Era is finished, i.e. after the ClusterEraPaid (formerly ClusterEraValidated) event for that Era is emitted. Otherwise the Payouts process will be broken, as it can last many blocks more after the actual validation.

Copy link

gitguardian bot commented Nov 20, 2024

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
9100662 Triggered Generic High Entropy Secret f52f5c0 node/service/chain-specs/example.json View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@khssnv
Copy link
Member Author

khssnv commented Nov 20, 2024

We should remove the cached data (activity aggregates) for an Era only after this payout process for this Era is finished, i.e. after the ClusterEraPaid (formerly ClusterEraValidated) event for that Era is emitted...

Thanks @yahortsaryk, as far as I see we don't use the data during and after the end_billing_report extrinsic call (which emits ClusterEraPaid event) so I put the deletion right after OCW sends it.

@khssnv khssnv force-pushed the feature/clear-ocw-validation-activities branch from 04d4ce0 to 57acdfd Compare November 26, 2024 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants