Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: don't simulate completed proposals #117

Open
mds1 opened this issue Sep 14, 2023 · 0 comments
Open

perf: don't simulate completed proposals #117

mds1 opened this issue Sep 14, 2023 · 0 comments

Comments

@mds1
Copy link
Contributor

mds1 commented Sep 14, 2023

Currently CI runs all proposals for every governor, even if that proposal was executed years ago. This is inefficient and often results in failed runs due to rate limits and other issues. It also means CI takes ~3 hours to run when it should only take a few minutes.

Instead, filter proposals to make sure they're not yet finalized before running the simulations. The https://github.com/bgd-labs/seatbelt-for-ghosts fork has implemented this as follows:

  1. Define the cache file, it's called ./proposal-states.json. It stores a simple map from proposal ID to the integer representing the current proposal state
  2. During the simulation loop, lookup the proposal ID in the cache file to check if the state is a terminal state. If not, simulate the proposal, otherwise skip simulation.
  3. Save off the new reports to ./reports folder

We can do something very similar, but since we support multiple governors we need one main tweak: Our cache file cannot just map from proposal ID to state, but instead must map from Governor address to proposal ID to state.

Since the cached reports live in the reports folder, they'll automatically be uploaded to workflow artifacts. Additionally, by removing the reports folder from the gitignore, reports can be easily hyperlinked to.

However, users must be careful never to manually commit changes to the report folder, as this should only be done by CI. It's worth seeing how the bgd-labs fork manages this. One potential solution is always running all simulations e.g. once per day to restore any accidentally overwritten proposals (this has the added benefit of retroactively updating reports when report structure changes, e.g. new formats or checks).

Similarly, we should probably save reports generated from local simulation to a different folder that can stay gitignored, such as local-reports

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant