Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Start poller after fast sync is completed #54

Closed
wants to merge 7 commits into from
Closed

Start poller after fast sync is completed #54

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
fd2b284
wip
lesterli Sep 15, 2024
f989ec1
fix test
bap2pecs Sep 15, 2024
0b67e2a
fix edge case
bap2pecs Sep 15, 2024
82fb6c4
nit
bap2pecs Sep 15, 2024
b1ab7ea
feat: batch processing non-finalised blocks (#57)
lesterli Sep 21, 2024
16e3318
update e2e test for multiple blocks in channel, use babylon
lesterli Sep 22, 2024
3f3464a
fix: update poller Stop for TestFastSync
lesterli Sep 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions finality-provider/service/chain_poller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,10 @@ func (cp *ChainPoller) Start(startHeight uint64) error {
}

func (cp *ChainPoller) Stop() error {
if !cp.IsRunning() {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without this, TestOpFpNoVotingPower will fail b/c it's trying to stop a poller that's not started

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this the same as !cp.isStarted.Swap(false)? It's just nil is returned instead of an error

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, now TestOpFpNoVotingPower will start the poller, it can be removed.

cp.logger.Info("the chain poller is not running")
return nil
}
if !cp.isStarted.Swap(false) {
return fmt.Errorf("the chain poller has already stopped")
}
Expand Down
31 changes: 28 additions & 3 deletions finality-provider/service/fp_instance.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,12 +116,22 @@ func (fp *FinalityProviderInstance) Start() error {
zap.String("pk", fp.GetBtcPkHex()), zap.Uint64("height", startHeight))

poller := NewChainPoller(fp.logger, fp.cfg.PollerConfig, fp.cc, fp.consumerCon, fp.metrics)
fp.poller = poller

if err := poller.Start(startHeight); err != nil {
return fmt.Errorf("failed to start the poller: %w", err)
// get the last finalized height
lastFinalizedBlock, err := fp.latestFinalizedBlockWithRetry()
if err != nil {
return err
}

fp.poller = poller
// Start the poller if fast sync is disabled or there's no finalized block
if (fp.cfg.FastSyncInterval == 0 || lastFinalizedBlock == nil) && !fp.poller.IsRunning() {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without lastFinalizedBlock == nil, babylon e2e TestFastSync will fail b/c tm.WaitForFpVoteCast(t, fpIns) will timeout

this is b/c in the code below, tryFastSync will not do anything

image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move !fp.poller.IsRunning() out to a new if condition? imo if !fp.poller.IsRunning() then we need to return err? Also if this if statement is false, when will the poller start?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this if statement is false, we start the poller after fast sync is finished.

// start poller after fast sync is finished
if !fp.poller.IsRunning() {
   err := fp.poller.Start(res.LastProcessedHeight + 1)
   if err != nil {
   	fp.logger.Error("failed to start the poller", zap.Error(err))
   	fp.reportCriticalErr(err)
   }
   continue
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we should add these check because up to here the fast sync is already finished and should be safe to start the poller. Can we have a workaround in e2e?

if err := fp.poller.Start(startHeight); err != nil {
fp.logger.Error("failed to start the poller", zap.Error(err))
fp.reportCriticalErr(err)
return err
}
}

fp.laggingTargetChan = make(chan uint64, 1)

Expand Down Expand Up @@ -275,6 +285,16 @@ func (fp *FinalityProviderInstance) finalitySigSubmissionLoop() {
zap.Uint64("last_processed_height", res.LastProcessedHeight),
)

// start poller after fast sync is finished
if !fp.poller.IsRunning() {
err := fp.poller.Start(res.LastProcessedHeight + 1)
if err != nil {
fp.logger.Error("failed to start the poller", zap.Error(err))
fp.reportCriticalErr(err)
}
continue
}
Comment on lines +223 to +230
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the start of the poller should be controlled within the sig submission loop. The poller should be already started when the loop is on


// inform the poller to skip to the next block of the last
// processed one
err := fp.poller.SkipToHeight(fp.GetLastProcessedHeight() + 1)
Expand Down Expand Up @@ -399,6 +419,11 @@ func (fp *FinalityProviderInstance) checkLaggingLoop() {
}

func (fp *FinalityProviderInstance) tryFastSync(targetBlockHeight uint64) (*FastSyncResult, error) {
fp.logger.Debug(
"trying fast sync",
zap.String("pk", fp.GetBtcPkHex()),
zap.Uint64("target_block_height", targetBlockHeight))

if fp.inSync.Load() {
return nil, fmt.Errorf("the finality-provider %s is already in sync", fp.GetBtcPkHex())
}
Expand Down
85 changes: 43 additions & 42 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ require (
github.com/babylonlabs-io/babylon-sdk/demo v0.0.0-20240814002132-55e711397a82
github.com/babylonlabs-io/finality-gadget v0.1.1
github.com/btcsuite/btcd v0.24.2
github.com/btcsuite/btcd/btcec/v2 v2.3.2
github.com/btcsuite/btcd/btcec/v2 v2.3.4
github.com/btcsuite/btcd/btcutil v1.1.5
github.com/btcsuite/btcwallet/walletdb v1.4.0
github.com/cometbft/cometbft v0.38.9
Expand All @@ -25,36 +25,40 @@ require (
github.com/cosmos/gogoproto v1.5.0
github.com/cosmos/relayer/v2 v2.5.2
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.3.0
github.com/ethereum/go-ethereum v1.14.7
github.com/ethereum/go-ethereum v1.14.8
github.com/gogo/protobuf v1.3.3
github.com/golang/mock v1.6.0
github.com/jessevdk/go-flags v1.5.0
github.com/jsternberg/zap-logfmt v1.3.0
github.com/juju/fslock v0.0.0-20160525022230-4d5c94c67b4b
github.com/lightningnetwork/lnd v0.16.4-beta.rc1
github.com/lightningnetwork/lnd/kvdb v1.4.1
github.com/prometheus/client_golang v1.19.1
github.com/prometheus/client_golang v1.20.2
github.com/spf13/cobra v1.8.1
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.9.0
github.com/urfave/cli v1.22.14
go.uber.org/atomic v1.10.0
go.uber.org/zap v1.27.0
google.golang.org/grpc v1.64.0
google.golang.org/protobuf v1.34.1
google.golang.org/protobuf v1.34.2
)

require (
github.com/BurntSushi/toml v1.4.0 // indirect
github.com/babylonlabs-io/babylon-sdk/x v0.0.0-20240814002132-55e711397a82 // indirect
github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd // indirect
github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792 // indirect
github.com/cockroachdb/fifo v0.0.0-20240606204812-0bbfbd93a7ce // indirect
github.com/cosmos/ibc-go/modules/apps/callbacks v0.2.1-0.20231113120333-342c00b0f8bd // indirect
github.com/cosmos/ibc-go/modules/light-clients/08-wasm v0.0.0-20240429153234-e1e6da7e4ead // indirect
github.com/ethereum/go-verkle v0.1.1-0.20240306133620-7d920df305f0 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/hashicorp/raft-boltdb/v2 v2.3.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/protolambda/ctxlock v0.1.0 // indirect
github.com/shamaton/msgpack/v2 v2.2.0 // indirect
github.com/wlynxg/anet v0.0.4 // indirect
)

require (
Expand All @@ -69,30 +73,27 @@ require (
github.com/dlclark/regexp2 v1.7.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dop251/goja v0.0.0-20230806174421-c933cf95e127 // indirect
github.com/elastic/gosigar v0.14.2 // indirect
github.com/elastic/gosigar v0.14.3 // indirect
github.com/ethereum-optimism/go-ethereum-hdwallet v0.1.3 // indirect
github.com/ethereum-optimism/optimism v1.7.7
github.com/ethereum-optimism/superchain-registry/superchain v0.0.0-20240812010938-34a43d577f74 // indirect
github.com/ethereum-optimism/superchain-registry/superchain v0.0.0-20240821192748-42bd03ba8313 // indirect
github.com/ferranbt/fastssz v0.1.2 // indirect
github.com/fjl/memsize v0.0.2 // indirect
github.com/flynn/noise v1.1.0 // indirect
github.com/francoispqt/gojay v1.2.13 // indirect
github.com/gballet/go-libpcsclite v0.0.0-20191108122812-4678299bea08 // indirect
github.com/go-sourcemap/sourcemap v2.1.3+incompatible // indirect
github.com/godbus/dbus/v5 v5.1.0 // indirect
github.com/gofrs/flock v0.8.1 // indirect
github.com/google/gopacket v1.1.19 // indirect
github.com/google/pprof v0.0.0-20240424215950-a892ee059fd6 // indirect
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 // indirect
github.com/graph-gophers/graphql-go v1.3.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-bexpr v0.1.11 // indirect
github.com/hashicorp/go-msgpack v0.5.5 // indirect
github.com/hashicorp/go-msgpack/v2 v2.1.1 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/golang-lru/arc/v2 v2.0.7 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/hashicorp/raft v1.7.0 // indirect
github.com/hashicorp/raft-boltdb v0.0.0-20231211162105-6c830fa4535e // indirect
github.com/holiman/billy v0.0.0-20240216141850-2abb0c79d3c4 // indirect
github.com/holiman/bloomfilter/v2 v2.0.3 // indirect
github.com/huin/goupnp v1.3.0 // indirect
Expand All @@ -109,10 +110,10 @@ require (
github.com/klauspost/cpuid/v2 v2.2.8 // indirect
github.com/koron/go-ssdp v0.0.4 // indirect
github.com/libp2p/go-flow-metrics v0.1.0 // indirect
github.com/libp2p/go-libp2p v0.35.0 // indirect
github.com/libp2p/go-libp2p v0.36.2 // indirect
github.com/libp2p/go-libp2p-asn-util v0.4.1 // indirect
github.com/libp2p/go-libp2p-mplex v0.9.0 // indirect
github.com/libp2p/go-libp2p-pubsub v0.11.0 // indirect
github.com/libp2p/go-libp2p-pubsub v0.12.0 // indirect
github.com/libp2p/go-libp2p-testing v0.12.0 // indirect
github.com/libp2p/go-mplex v0.7.0 // indirect
github.com/libp2p/go-msgio v0.3.0 // indirect
Expand All @@ -138,30 +139,30 @@ require (
github.com/multiformats/go-multistream v0.5.0 // indirect
github.com/multiformats/go-varint v0.0.7 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/onsi/ginkgo/v2 v2.19.0 // indirect
github.com/onsi/ginkgo/v2 v2.20.0 // indirect
github.com/onsi/gomega v1.34.1 // indirect
github.com/opencontainers/runtime-spec v1.2.0 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect
github.com/peterh/liner v1.1.1-0.20190123174540-a2c9a5303de7 // indirect
github.com/pion/datachannel v1.5.6 // indirect
github.com/pion/dtls/v2 v2.2.11 // indirect
github.com/pion/ice/v2 v2.3.24 // indirect
github.com/pion/interceptor v0.1.29 // indirect
github.com/pion/datachannel v1.5.8 // indirect
github.com/pion/dtls/v2 v2.2.12 // indirect
github.com/pion/ice/v2 v2.3.34 // indirect
github.com/pion/interceptor v0.1.30 // indirect
github.com/pion/logging v0.2.2 // indirect
github.com/pion/mdns v0.0.12 // indirect
github.com/pion/randutil v0.1.0 // indirect
github.com/pion/rtcp v1.2.14 // indirect
github.com/pion/rtp v1.8.6 // indirect
github.com/pion/sctp v1.8.16 // indirect
github.com/pion/rtp v1.8.9 // indirect
github.com/pion/sctp v1.8.33 // indirect
github.com/pion/sdp/v3 v3.0.9 // indirect
github.com/pion/srtp/v2 v2.0.18 // indirect
github.com/pion/srtp/v2 v2.0.20 // indirect
github.com/pion/stun v0.6.1 // indirect
github.com/pion/transport/v2 v2.2.5 // indirect
github.com/pion/transport/v2 v2.2.10 // indirect
github.com/pion/turn/v2 v2.1.6 // indirect
github.com/pion/webrtc/v3 v3.2.40 // indirect
github.com/pion/webrtc/v3 v3.3.0 // indirect
github.com/quic-go/qpack v0.4.0 // indirect
github.com/quic-go/quic-go v0.44.0 // indirect
github.com/quic-go/quic-go v0.46.0 // indirect
github.com/quic-go/webtransport-go v0.8.0 // indirect
github.com/raulk/go-watchdog v1.3.0 // indirect
github.com/rivo/uniseg v0.4.4 // indirect
Expand All @@ -171,10 +172,10 @@ require (
github.com/urfave/cli/v2 v2.27.4 // indirect
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 // indirect
github.com/yusufpapurcu/wmi v1.2.3 // indirect
go.uber.org/dig v1.17.1 // indirect
go.uber.org/fx v1.21.1 // indirect
go.uber.org/dig v1.18.0 // indirect
go.uber.org/fx v1.22.2 // indirect
go.uber.org/mock v0.4.0 // indirect
lukechampine.com/blake3 v1.2.1 // indirect
lukechampine.com/blake3 v1.3.0 // indirect
)

require (
Expand Down Expand Up @@ -218,7 +219,7 @@ require (
github.com/cockroachdb/apd/v2 v2.0.2 // indirect
github.com/cockroachdb/errors v1.11.3 // indirect
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect
github.com/cockroachdb/pebble v1.1.0 // indirect
github.com/cockroachdb/pebble v1.1.2 // indirect
github.com/cockroachdb/redact v1.1.5 // indirect
github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 // indirect
github.com/cometbft/cometbft-db v0.9.1 // indirect
Expand Down Expand Up @@ -259,7 +260,7 @@ require (
github.com/go-kit/kit v0.12.0 // indirect
github.com/go-kit/log v0.2.1 // indirect
github.com/go-logfmt/logfmt v0.6.0 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect
Expand All @@ -279,7 +280,7 @@ require (
github.com/googleapis/gax-go/v2 v2.12.3 // indirect
github.com/gorilla/handlers v1.5.2 // indirect
github.com/gorilla/mux v1.8.1 // indirect
github.com/gorilla/websocket v1.5.1 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
Expand Down Expand Up @@ -331,7 +332,7 @@ require (
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mholt/archiver/v3 v3.5.0 // indirect
github.com/miekg/dns v1.1.58 // indirect
github.com/miekg/dns v1.1.62 // indirect
github.com/minio/highwayhash v1.0.2 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
Expand All @@ -350,8 +351,8 @@ require (
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.52.2 // indirect
github.com/prometheus/procfs v0.13.0 // indirect
github.com/prometheus/common v0.55.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 // indirect
github.com/rogpeppe/go-internal v1.12.0 // indirect
Expand Down Expand Up @@ -403,16 +404,16 @@ require (
go.opentelemetry.io/proto/otlp v0.9.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/crypto v0.26.0 // indirect
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
golang.org/x/mod v0.19.0 // indirect
golang.org/x/net v0.27.0 // indirect
golang.org/x/oauth2 v0.20.0 // indirect
golang.org/x/exp v0.0.0-20240808152545-0cdaa3abc0fa // indirect
golang.org/x/mod v0.20.0 // indirect
golang.org/x/net v0.28.0 // indirect
golang.org/x/oauth2 v0.21.0 // indirect
golang.org/x/sync v0.8.0 // indirect
golang.org/x/sys v0.23.0 // indirect
golang.org/x/sys v0.24.0 // indirect
golang.org/x/term v0.23.0 // indirect
golang.org/x/text v0.17.0 // indirect
golang.org/x/time v0.6.0 // indirect
golang.org/x/tools v0.23.0 // indirect
golang.org/x/tools v0.24.0 // indirect
google.golang.org/api v0.171.0 // indirect
google.golang.org/genproto v0.0.0-20240227224415-6ceb2ff114de // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240513163218-0867130af1f8 // indirect
Expand All @@ -432,7 +433,7 @@ require (
modernc.org/sqlite v1.20.3 // indirect
modernc.org/strutil v1.1.3 // indirect
modernc.org/token v1.0.1 // indirect
nhooyr.io/websocket v1.8.17 // indirect
nhooyr.io/websocket v1.8.6 // indirect
pgregory.net/rapid v1.1.0 // indirect
rsc.io/tmplfunc v0.0.3 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
Expand All @@ -442,9 +443,9 @@ replace (
// use cosmos fork of keyring
github.com/99designs/keyring => github.com/cosmos/keyring v1.2.0
github.com/babylonlabs-io/babylon => github.com/babylonlabs-io/babylon-private v0.9.0-rc.3.0.20240801001431-74a24c962ce2
github.com/cockroachdb/pebble => github.com/cockroachdb/pebble v0.0.0-20231018212520-f6cde3fc2fa4
github.com/ethereum-optimism/optimism => github.com/babylonlabs-io/optimism v1.8.1-0.20240829090216-6b49c6e93dfe
github.com/ethereum/go-ethereum => github.com/ethereum-optimism/op-geth v1.101407.0-rc.1.0.20240812224053-8d99ca68bb1a
github.com/btcsuite/btcd/btcec/v2 => github.com/btcsuite/btcd/btcec/v2 v2.3.2
github.com/ethereum-optimism/optimism => github.com/babylonlabs-io/optimism v1.8.1-0.20240901154247-a38e38a679ab
github.com/ethereum/go-ethereum v1.14.8 => github.com/ethereum-optimism/op-geth v1.101408.0-rc.4.0.20240827042333-110c433a2469
github.com/gogo/protobuf => github.com/regen-network/protobuf v1.3.3-alpha.regen.1
github.com/syndtr/goleveldb => github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7
)
Loading
Loading