Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

loop print db stats and announce hashes failed and don't sync #227

Open
blockpi019 opened this issue Nov 21, 2024 · 3 comments
Open

loop print db stats and announce hashes failed and don't sync #227

blockpi019 opened this issue Nov 21, 2024 · 3 comments

Comments

@blockpi019
Copy link

blockpi019 commented Nov 21, 2024

System information

Network: mainnet

if you are running a local node, please provide the following information:
op-node version: op-node v0.5.1
op-geth version: op-geth v0.5.2
OS & Version: Linux Ubuntu 22.04.4 LTS

Expected behaviour

Normal synchronisation and access to block information

Actual behaviour

the geth client log: loop print db stats

Steps to reproduce the behaviour

the node don't sync at height 32984678. and the geth client log is loop print db stats
the log is below:
Nov 21 12:14:50 op-geth[571215]: INFO [11-21|12:14:50.510] announced transaction request timeout hashes= num=0
Nov 21 12:14:52 op-geth[571215]: INFO [11-21|12:14:52.616] loop print db stats database=/node/full/opbnb/geth/chaindata comp_time=1,476,091,229 write_delay_count=0 write_delay_time=0 non_level0_comp_count=57 level0_comp_count=7
Nov 21 12:14:55 op-geth[571215]: INFO [11-21|12:14:55.512] announced transaction request timeout hashes= num=0
Nov 21 12:14:55 op-geth[571215]: INFO [11-21|12:14:55.616] loop print db stats database=/node/full/opbnb/geth/chaindata comp_time=1,476,091,229 write_delay_count=0 write_delay_time=0 non_level0_comp_count=57 level0_comp_count=7
Nov 21 12:14:58 op-geth[571215]: INFO [11-21|12:14:58.616] loop print db stats database=/node/full/opbnb/geth/chaindata comp_time=1,476,091,229 write_delay_count=0 write_delay_time=0 non_level0_comp_count=57 level0_comp_count=7
Nov 21 12:15:00 op-geth[571215]: INFO [11-21|12:15:00.291] announced transaction request timeout hashes= num=0
Nov 21 12:15:01 op-geth[571215]: INFO [11-21|12:15:01.616] loop print db stats database=/node/full/opbnb/geth/chaindata comp_time=1,476,091,229 write_delay_count=0 write_delay_time=0 non_level0_comp_count=57 level0_comp_count=7
Nov 21 12:15:04 op-geth[571215]: INFO [11-21|12:15:04.618] loop print db stats database=/node/full/opbnb/geth/chaindata comp_time=1,476,091,229 write_delay_count=0 write_delay_time=0 non_level0_comp_count=57 level0_comp_count=7
Nov 21 12:15:04 op-geth[571215]: WARN [11-21|12:15:04.678] Announce hashes failed id=f2e5c40cf638856e conn=dyndial peerId=f2e5c40cf638856e027e4fcd29a51d187e4b0c68a68224cdcf58d115c0c3bdf0 peerIP=3.10.19.215 lost=3200 hashes=0x873c89adc23b183926dc31a02d10b63fa7e6ee9fa3035e6cbc379b0e514af006,0xde022dac0b025695b2d73ba6ffd7451a88845d2b858e6f12ee3c59c535de70e7,0xe6d6a26c8f5772626772654ff9ba86d160db2aa242a08fba642bbfaab7000f96,0xa8f7b592ee8ada9d36b12c2f513c13a4262087147e487764673c705ba92a0742,0x45b0649945fbbcc1f596826d004e8c2b703466b9914132d5efe266c31f4af6e7,0xc3ec1190ef00c9dcf9d6d1f05ae532641f655f7ad0c68a80c81f66e154d1d4c0,0x17af014efae36376d99eb58feec0dcca0e051dce52ddf31130519eccb26f6d34,0x0d78bb47fe9df8e0f1679f298b92a339c8e1e6d1e3ece0975aade32ee2ff2d2d,0x060da201a2fb04317a11d4631b655deabf219562c9ce3ffd0133d54b310327e9,0xe06d20346d9c0bf833ddb261a47d62f23fac3d392c915620f4693e4497157f5d,0xd8d1c33fd7e176eec622499e689b517bab0ba52cdd6d5709014f3c2e6ffc062a,0x0e8152d2496d16bd30f0613951390a22ee365c2624ecf03a75bb7bffb9e8bba3,0x1ba1991bc49229ef84138a319123258a9b7cb6d72020bdeca2bedf098931b771,0x87ebc7721c339cd5b871cbbf8d16d57f9d1aa67154d813ca046f887fa3cc6653,0xfad49b203513ae40a2099ce542ed3e709ee06a12be5f5a98baa8c9ac01f3e94b,0x336e00a1e062ed787fbd78f28888a0d11d6d96ef12bc20b7953d49ba9944743c,0x17e11a38d7f0c5458e715f0c6f2544f05baad4c8aa82d0b575666be0a943e5f0,0x47603bafc8e90834e0fb8a8e5ed54537bd4f062c35c94361b2407fd942aa5bc5,0x6c9f94cf4618d32034f5544b05a8531f7e0b134fee21b9b72bfa9a936bdf7fd4,0x7b18eba2f519f12ab48c74649797fceebe8832e9b33a765ec0ebcfdea294e749,......,0x851df1ac6a9aaf7d744afdddbd447556513bb1388966b228b687a517330cf6f6,0xc967bcea40ddb621eda1859e5375516500637faf640e892f9cc5aabe85006a83,0xd9e681418a1bf29177c86bf3efdd5c5ce9840df1e5a66fd2f7abdff27f91727a,0x1b4ad47df07cbf918aadbd22eb8b8443d469b5efe6eab0b1fa606b694117a2fb,0x8919c91b3a3c8b9cda3cdaf86d69b1bacc32774dd907f17ba81eea6ebe41320d,0x13301ce59d04f5ec47a10be3458e8f720932900a7575726113ea6a7436444a76,0xd31754692ddf134eac78893606272552f40095bf3aa87d760dc40693896254b3,0xe75ddbca19d0b8ce9b318e2e9afcfe103f8f806820b1e173c7927387cec2606c,0x48411592c9b1aa54f3edc4bd416d1e55

Backtrace

[backtrace]

When submitting logs: please submit them as text and not screenshots.

@welkin22
Copy link
Contributor

welkin22 commented Nov 21, 2024

@blockpi019 Please provide more information, such as the op-node logs and the startup parameters for op-node and op-geth.
Additionally, you can enable the debug log of op-geth by using the --verbosity=4 parameter, which will provide more information to help identify the issue.

@blockpi019
Copy link
Author

startup parameters:
op-geth:
/node/full/opbnb/bin/op-geth --datadir=/node/full/opbnb/geth --state.scheme=path --db.engine=pebble --http --http.corsdomain=* --http.vhosts=* --http.addr=0.0.0.0 --http.port=8445 --http.api=eth,web3,net,debug,txpool --ws --ws.addr=0.0.0.0 --ws.port=8446 --ws.origins=* --ws.api=eth,web3,net,debug,txpool --rpc.gascap=600000000 --syncmode=full --gcmode=full --maxpeers=100 --networkid=204 --txpool.globalslots=1000000 --txpool.globalqueue=3000000 --txpool.accountqueue=200 --txpool.accountslots=200 --cache 32000 --cache.preimages --allow-insecure-unlock --authrpc.port=8447 --authrpc.vhosts=* --authrpc.jwtsecret=/node/full/opbnb/jwt.txt --rollup.sequencerhttp=https://opbnb-mainnet-rpc.bnbchain.org --bootnodes=enr:-KO4QHs5qh_kPFcjMgqkuN9dbxXT4C5Cjad4SAheaUxveCbJQ3XdeMMDHeHilHyqisyYQAByfdhzyKAdUp2SvyzWeBqGAYvRDf80g2V0aMfGhHFtSjqAgmlkgnY0gmlwhDaykUmJc2VjcDI1NmsxoQJUevTL3hJwj21IT2GC6VaNqVQEsJFPtNtO-ld5QTNCfIRzbmFwwIN0Y3CCdl-DdWRwgnZf,enr:-KO4QKIByq-YMjs6IL2YCNZEmlo3dKWNOy4B6sdqE3gjOrXeKdNbwZZGK_JzT1epqCFs3mujjg2vO1lrZLzLy4Rl7PyGAYvRA8bEg2V0aMfGhHFtSjqAgmlkgnY0gmlwhDbjSM6Jc2VjcDI1NmsxoQNQhJ5pqCPnTbK92gEc2F98y-u1OgZVAI1Msx-UiHezY4RzbmFwwIN0Y3CCdl-DdWRwgnZf --verbosity=4

op-node:
/node/full/opbnb/bin/op-node --l1.trustrpc --sequencer.l1-confs=15 --verifier.l1-confs=15 --l1.http-poll-interval 3s --l1.epoch-poll-interval 45s --l1.rpc-max-batch-size 20 --rollup.config=/node/full/opbnb/bin/rollup.json --rpc.addr=0.0.0.0 --rpc.port=9445 --p2p.sync.req-resp --p2p.listen.ip=0.0.0.0 --p2p.listen.tcp=9446 --p2p.listen.udp=9447 --snapshotlog.file=/node/full/opbnb/snapshot.log --p2p.bootnodes=enr:-J24QA9sgVxbZ0KoJ7-1gx_szfc7Oexzz7xL2iHS7VMHGj2QQaLc_IQZmFthywENgJWXbApj7tw7BiouKDOZD4noWEWGAYppffmvgmlkgnY0gmlwhDbjSM6Hb3BzdGFja4PMAQCJc2VjcDI1NmsxoQKetGQX7sXd4u8hZr6uayTZgHRDvGm36YaryqZkgnidS4N0Y3CCIyuDdWRwgiMs,enr:-J24QPSZMaGw3NhO6Ll25cawknKcOFLPjUnpy72HCkwqaHBKaaR9ylr-ejx20INZ69BLLj334aEqjNHKJeWhiAdVcn-GAYv28FmZgmlkgnY0gmlwhDTDWQOHb3BzdGFja4PMAQCJc2VjcDI1NmsxoQJ-_5GZKjs7jaB4TILdgC8EwnwyL3Qip89wmjnyjvDDwoN0Y3CCIyuDdWRwgiMs --rpc.enable-admin --l1=https://rpc-bsc.48.club --l2=http://127.0.0.1:8447 --l2.jwt-secret=/node/full/opbnb/jwt.txt --l2.engine-sync=true --l2.skip-sync-start-check=true

@welkin22
Copy link
Contributor

startup parameters: op-geth: /node/full/opbnb/bin/op-geth --datadir=/node/full/opbnb/geth --state.scheme=path --db.engine=pebble --http --http.corsdomain=* --http.vhosts=* --http.addr=0.0.0.0 --http.port=8445 --http.api=eth,web3,net,debug,txpool --ws --ws.addr=0.0.0.0 --ws.port=8446 --ws.origins=* --ws.api=eth,web3,net,debug,txpool --rpc.gascap=600000000 --syncmode=full --gcmode=full --maxpeers=100 --networkid=204 --txpool.globalslots=1000000 --txpool.globalqueue=3000000 --txpool.accountqueue=200 --txpool.accountslots=200 --cache 32000 --cache.preimages --allow-insecure-unlock --authrpc.port=8447 --authrpc.vhosts=* --authrpc.jwtsecret=/node/full/opbnb/jwt.txt --rollup.sequencerhttp=https://opbnb-mainnet-rpc.bnbchain.org --bootnodes=enr:-KO4QHs5qh_kPFcjMgqkuN9dbxXT4C5Cjad4SAheaUxveCbJQ3XdeMMDHeHilHyqisyYQAByfdhzyKAdUp2SvyzWeBqGAYvRDf80g2V0aMfGhHFtSjqAgmlkgnY0gmlwhDaykUmJc2VjcDI1NmsxoQJUevTL3hJwj21IT2GC6VaNqVQEsJFPtNtO-ld5QTNCfIRzbmFwwIN0Y3CCdl-DdWRwgnZf,enr:-KO4QKIByq-YMjs6IL2YCNZEmlo3dKWNOy4B6sdqE3gjOrXeKdNbwZZGK_JzT1epqCFs3mujjg2vO1lrZLzLy4Rl7PyGAYvRA8bEg2V0aMfGhHFtSjqAgmlkgnY0gmlwhDbjSM6Jc2VjcDI1NmsxoQNQhJ5pqCPnTbK92gEc2F98y-u1OgZVAI1Msx-UiHezY4RzbmFwwIN0Y3CCdl-DdWRwgnZf --verbosity=4

op-node: /node/full/opbnb/bin/op-node --l1.trustrpc --sequencer.l1-confs=15 --verifier.l1-confs=15 --l1.http-poll-interval 3s --l1.epoch-poll-interval 45s --l1.rpc-max-batch-size 20 --rollup.config=/node/full/opbnb/bin/rollup.json --rpc.addr=0.0.0.0 --rpc.port=9445 --p2p.sync.req-resp --p2p.listen.ip=0.0.0.0 --p2p.listen.tcp=9446 --p2p.listen.udp=9447 --snapshotlog.file=/node/full/opbnb/snapshot.log --p2p.bootnodes=enr:-J24QA9sgVxbZ0KoJ7-1gx_szfc7Oexzz7xL2iHS7VMHGj2QQaLc_IQZmFthywENgJWXbApj7tw7BiouKDOZD4noWEWGAYppffmvgmlkgnY0gmlwhDbjSM6Hb3BzdGFja4PMAQCJc2VjcDI1NmsxoQKetGQX7sXd4u8hZr6uayTZgHRDvGm36YaryqZkgnidS4N0Y3CCIyuDdWRwgiMs,enr:-J24QPSZMaGw3NhO6Ll25cawknKcOFLPjUnpy72HCkwqaHBKaaR9ylr-ejx20INZ69BLLj334aEqjNHKJeWhiAdVcn-GAYv28FmZgmlkgnY0gmlwhDTDWQOHb3BzdGFja4PMAQCJc2VjcDI1NmsxoQJ-_5GZKjs7jaB4TILdgC8EwnwyL3Qip89wmjnyjvDDwoN0Y3CCIyuDdWRwgiMs --rpc.enable-admin --l1=https://rpc-bsc.48.club --l2=http://127.0.0.1:8447 --l2.jwt-secret=/node/full/opbnb/jwt.txt --l2.engine-sync=true --l2.skip-sync-start-check=true

Hi @blockpi019, I checked your startup parameters and didn't find any issues, except that l2.engine-sync=true is a deprecated configuration. It is now recommended to use --syncmode=execution-layer. It is not the cause of the problem, and we need to investigate further.
Looping to print db stats should not be the key issue, as it is supposed to print.
I suggest you first check the status of the p2p peers, for example, use the following API to get the number of peers connected in op-geth:

curl -X POST -H "Content-Type: application/json" -d '[{"jsonrpc": "2.0","id": 1,"method": "net_peerCount", "params": []}]' http://127.0.0.1:8445

In addition, we need to know the current status of the op-node, so you need to observe the latest logs of the op-node. If the synchronization is normal, it should continuously print the following logs:

Optimistically inserting unsafe L2 execution payload to drive EL sync

If the op-node status is abnormal, there might be some error logs. Please provide them if convenient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants