History Replay

Support for partial history replay was implemented in PR #1524:

Lets us do replays of any part of the node's history, on any chains we want. The upper bound is fastForwardBlockHeightLimit, the lower bound is initialBlockHeightLimit. syncPactChains is a new option letting us set which chains we want to replay on. Implementation details follow.

If onlySyncPact is set, the cut store is now treated as read-only, and the cut db and Pact do a "fast forward" replay after the initial reset. By default that fast forward goes to the highest stored cut. There is also a new setting fastForwardBlockHeightLimit which is settable from the command line and config file, which can be used to limit the fast forward block height.

e.g. with onlySyncPact set, if you set initialBlockHeightLimit to 1_000000, and the node's existing highest cut is at 1_500000, and the node starts:

the node rewinds the Pact state to the blocks at height 1_000000, then replays Pact state to the blocks at height 1_500000, then exits. If you were to set fastForwardBlockHeightLimit to 1_200000, then the replay would only go to 1_200000.

A subsequent start on the same database without onlySyncPact or initialBlockHeightLimit set will retain the cut at 1_500000, and use it to replay to 1_500000, then begin normal operation.

Relevant Configuration:

The relevant configuration settings (with the respective default values) are:

chainweb:
  onlySyncPact: false
  validateHashesOnReplay: false
  syncPactChains: null
  cuts:
    fastForwardBlockHeightLimit: null
    pruneChainDatabase: none
    initialBlockHeightLimit: null

The respective command line arguments are:

• --only-sync-pact
• --validateHashesOnReplay
• --sync-pact-chains <JSON List>
• --fast-forward-block-height-limit <INT>
• --prune-chain-database <none|headers|headers-checked|full>
• --initial-block-height-limit <INT>

The overall behavior depends on the value for onlySyncPact.

Pact Replay Mode

In this mode only pact state is updated. The Rocksdb is opened in read-only mode. In particular the latest cut is retrained.

chainweb:
  onlySyncPact: true

  # This should be set to `true` in this mode.
  # Double check that the computed payload hash matches the payload hash
  # that is stored in the corresponding block header.
  validateHashesOnReplay: true

  # What chains to sync (null means all chains)
  syncPactChains: null

  cuts:

    # Stop replay of pact state at this height
    fastForwardBlockHeightLimit: null

    # Set this to 'headers-checked' for validating the complete Merkle Tree
    # (starting with the latest cut going down all the way to the genesis
    # headers)
    pruneChainDatabase: none

    # Start a fast forward pact replay at this height.
    initialBlockHeightLimit: null

Normal Node Operation

In this mode the Rocksdb is writeable. The node synchronizes with network after startup and start validating new cuts and participates in consensus.

chainweb:
  onlySyncPact: false

  # This can be set to `false` in this mode.
  validateHashesOnReplay: false

  # Ignored
  syncPactChains: null

  cuts:

    # Ignored
    fastForwardBlockHeightLimit: null

    # Should be 'none' during normal operation.
    pruneChainDatabase: none

    # Reset initial cut to this height on startup. Delete all cuts that are
    # larger than this value. The node will still have access to previously
    # stored paylaods and block headers, because only cuts are deleted.
    initialBlockHeightLimit: null