Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Slave 同步异常 #2949

Closed
loverkoo opened this issue Nov 13, 2024 · 11 comments
Closed

Slave 同步异常 #2949

loverkoo opened this issue Nov 13, 2024 · 11 comments
Labels
☢️ Bug Something isn't working

Comments

@loverkoo
Copy link

loverkoo commented Nov 13, 2024 •
edited
Loading

Is this a regression?

Yes

Description

pika4.0.1版本，新建slave，出现全量不同步直接跳过，做增量同步。
在3.5.5版本没有此问题。

I20241113 15:01:30.381704 1 pika_server.cc:214] Pika Server going to start
I20241113 15:01:30.381721 1 rsync_server.cc:48] start RsyncServer ...
I20241113 15:01:30.381942 1 rsync_server.cc:60] RsyncServer started ...
I20241113 15:03:24.422560 1044 pika_server.cc:1797] clear cache start...
I20241113 15:03:24.422659 1044 pika_server.cc:1801] clear cache finish
I20241113 15:03:24.460886 1039 pika_repl_client.cc:199] Try Send Meta Sync Request to Master (10.42.128.221:9221)
I20241113 15:03:24.463303 1009 pika_server.cc:452] Mark try connect finish
I20241113 15:03:24.463346 1009 pika_repl_client_conn.cc:155] Finish to handle meta sync response
I20241113 15:03:24.603641 1009 pika_repl_client_conn.cc:234] DB: db0 Need To Try DBSync
I20241113 15:03:24.603732 1009 pika_repl_client_conn.cc:231] DB: db1 TrySync Ok
I20241113 15:03:24.661801 1009 pika_repl_client_conn.cc:186] DB: db0 Need Wait To Sync
I20241113 15:03:24.662063 1009 base_conf.cc:345] ret OK
W20241113 15:03:24.761471 1039 pika_rm.cc:510] Slave DB: db0 Activating Rsync ... (retry count:0)
W20241113 15:03:24.762320 1045 rsync_client.h:228] rsync response error
W20241113 15:03:24.762360 1039 rsync_client.cc:382] IO error: kRsyncMeta request failed! db is not exist or doing bgsave, retries:0
I20241113 15:03:25.803998 1039 rsync_client.cc:387] receive rsync meta infos, snapshot_uuid: bbba7f8908b9d46e35e4813bc869971dfiles count: 13
W20241113 15:03:25.804076 1039 rsync_client.cc:403] DUMP_META_DATA not exist
I20241113 15:03:25.804704 1039 rsync_client.cc:336] copy meta data done, db name: db0 snapshot_uuid: bbba7f8908b9d46e35e4813bc869971d file count: 13 expired file count: 0 local file count: 0 remote file count: 13 remote snapshot_uuid: bbba7f8908b9d46e35e4813bc869971d local snapshot_uuid: file_set_: 13
W20241113 15:03:25.804741 1039 rsync_client.cc:346] file_set: 0/000052.log
W20241113 15:03:25.804761 1039 rsync_client.cc:346] file_set: 0/CURRENT
W20241113 15:03:25.804780 1039 rsync_client.cc:346] file_set: 0/MANIFEST-000053
W20241113 15:03:25.804798 1039 rsync_client.cc:346] file_set: 0/OPTIONS-000055
W20241113 15:03:25.804816 1039 rsync_client.cc:346] file_set: 1/000052.log
W20241113 15:03:25.804836 1039 rsync_client.cc:346] file_set: 1/CURRENT
W20241113 15:03:25.804853 1039 rsync_client.cc:346] file_set: 1/MANIFEST-000053
W20241113 15:03:25.804872 1039 rsync_client.cc:346] file_set: 1/OPTIONS-000055
W20241113 15:03:25.804890 1039 rsync_client.cc:346] file_set: 2/000052.log
W20241113 15:03:25.804908 1039 rsync_client.cc:346] file_set: 2/CURRENT
W20241113 15:03:25.804926 1039 rsync_client.cc:346] file_set: 2/MANIFEST-000053
W20241113 15:03:25.804945 1039 rsync_client.cc:346] file_set: 2/OPTIONS-000055
W20241113 15:03:25.804963 1039 rsync_client.cc:346] file_set: info
I20241113 15:03:25.804983 1039 rsync_client.cc:76] RsyncClient recover success
I20241113 15:03:25.805119 1046 rsync_client.cc:90] RsyncClient begin to copy remote files
I20241113 15:03:25.805392 1047 rsync_client.cc:40] copy remote file, filename: 0/000052.log
I20241113 15:03:25.805444 1048 rsync_client.cc:40] copy remote file, filename: 0/CURRENT
I20241113 15:03:25.805521 1049 rsync_client.cc:40] copy remote file, filename: 0/MANIFEST-000053
I20241113 15:03:25.805583 1050 rsync_client.cc:40] copy remote file, filename: 0/OPTIONS-000055
I20241113 15:03:25.862306 1048 rsync_client.cc:40] copy remote file, filename: 1/CURRENT
I20241113 15:03:25.862324 1047 rsync_client.cc:40] copy remote file, filename: 1/000052.log
I20241113 15:03:25.872570 1049 rsync_client.cc:40] copy remote file, filename: 1/MANIFEST-000053
I20241113 15:03:25.874122 1050 rsync_client.cc:40] copy remote file, filename: 1/OPTIONS-000055
I20241113 15:03:25.875551 1048 rsync_client.cc:40] copy remote file, filename: 2/CURRENT
I20241113 15:03:25.876278 1047 rsync_client.cc:40] copy remote file, filename: 2/000052.log
I20241113 15:03:25.878705 1049 rsync_client.cc:40] copy remote file, filename: 2/MANIFEST-000053
I20241113 15:03:25.879336 1050 rsync_client.cc:40] copy remote file, filename: 2/OPTIONS-000055
I20241113 15:03:25.881747 1047 rsync_client.cc:40] copy remote file, filename: info
I20241113 15:03:25.883343 1048 rsync_client.cc:53] work_thread index: 1 copy remote files done
I20241113 15:03:25.884796 1049 rsync_client.cc:53] work_thread index: 2 copy remote files done
I20241113 15:03:25.884938 1050 rsync_client.cc:53] work_thread index: 3 copy remote files done
I20241113 15:03:25.886230 1047 rsync_client.cc:53] work_thread index: 0 copy remote files done
I20241113 15:03:25.886305 1046 rsync_client.cc:153] RsyncClient copy remote files done
I20241113 15:03:25.905236 1039 pika_db.cc:500] DB: db0 Information from dbsync info, master_ip: 10.42.128.221, master_port: 9221, filenum: 29, offset: 95216551, term: 0, index: 0
I20241113 15:03:25.905330 1039 pika_db.cc:555] DB: db0, Prepare change db from: ./data/db/db0_bak
I20241113 15:03:25.956422 1039 pika_db.cc:576] DB: db0, Change db success
I20241113 15:03:26.070235 1039 base_conf.cc:345] ret OK
I20241113 15:03:26.211719 1009 pika_repl_client_conn.cc:231] DB: db0 TrySync Ok

Please provide a link to a minimal reproduction of the bug

No response

Screenshots or videos

No response

Please provide the version you discovered this bug in (check about page for version information)

版本：4.0.1
linux：ubuntu 22.04

Anything else?

此问题在之前的issue中被提到，但实际未修复
之前issue如下：
#2742
@loverkoo loverkoo added the ☢️ Bug Something isn't working label Nov 13, 2024
@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.

Title: Slave synchronization exception

@wangshao1
Copy link
Collaborator

从日志上来看，master下没有sst文件。如果能复现的话，麻烦贴下master和slave端的日志，db目录下sst文件个数，dump目录下sst文件个数。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.

Judging from the log, there is no sst file under the master. If it can be reproduced, please post the master and slave logs, the number of sst files in the db directory, and the number of sst files in the dump directory.

@loverkoo
Copy link
Author

从日志上来看，master下没有sst文件。如果能复现的话，麻烦贴下master和slave端的日志，db目录下sst文件个数，dump目录下sst文件个数。

日志
master

I20241122 14:10:06.193110 1731 dispatch_thread.cc:151] accept new conn 85:10.42.128.192:38248:0
I20241122 14:10:06.193171 1731 dispatch_thread.cc:159] find worker(0), refresh the last_thread_ to 0
I20241122 14:10:06.193827 1711 pika_repl_server_conn.cc:38] Receive MetaSync, Slave ip: 10.42.128.192, Slave port:9221
I20241122 14:10:06.193852 1711 pika_server.cc:642] Add New Slave, 10.42.128.192:9221
I20241122 14:10:06.194515 1731 dispatch_thread.cc:151] accept new conn 86:10.42.128.192:38264:0
I20241122 14:10:06.194540 1731 dispatch_thread.cc:159] find worker(0), refresh the last_thread_ to 0
I20241122 14:10:06.194819 1731 dispatch_thread.cc:151] accept new conn 86:10.42.128.192:38270:0
I20241122 14:10:06.194836 1731 dispatch_thread.cc:159] find worker(0), refresh the last_thread_ to 0
I20241122 14:10:06.337050 1713 pika_repl_server_conn.cc:114] Receive Trysync, Slave ip: 10.42.128.192, Slave port:9221, DB: db0, filenum: 0, pro_offset: 0
I20241122 14:10:06.337050 1712 pika_repl_server_conn.cc:114] Receive Trysync, Slave ip: 10.42.128.192, Slave port:9221, DB: db1, filenum: 0, pro_offset: 0
I20241122 14:10:06.337113 1713 pika_repl_server_conn.cc:200] DB: db0 binlog has been purged, may need full sync
I20241122 14:10:06.337239 1712 pika_rm.cc:69] Add Slave Node, db: (db1), ip_port: 10.42.128.192:9221
I20241122 14:10:06.337267 1712 pika_repl_server_conn.cc:156] DB: db1 TrySync Success, Session: 2
I20241122 14:10:06.394114 1713 pika_repl_server_conn.cc:243] Handle DBSync Request
I20241122 14:10:06.394176 1713 pika_rm.cc:69] Add Slave Node, db: (db0), ip_port: 10.42.128.192:9221
I20241122 14:10:06.394201 1713 pika_repl_server_conn.cc:264] DB: db0 Handle DBSync Request Success, Session: 2
I20241122 14:10:06.417064 1738 pika_db.cc:289] db0 after prepare bgsave
I20241122 14:10:06.417088 1738 pika_db.cc:292] db0 bgsave_info: path=./data/dump/20241122/db0, filenum=29, offset=95216551
I20241122 14:10:06.470862 1738 pika_db.cc:302] db0 create new backup finished.
W20241122 14:10:06.494846 1734 rsync_server.cc:241] fd: 87 ip_port: 10.42.128.192 connection accepted
I20241122 14:10:06.495273 1735 rsync_server.cc:146] Rsync Meta request, snapshot_uuid: d1ddbf4abaa8e8b966c12047f486c403 files count: 13 file list:
I20241122 14:10:06.495299 1735 rsync_server.cc:149] rsync snapshot file: 0/CURRENT
I20241122 14:10:06.495306 1735 rsync_server.cc:149] rsync snapshot file: 0/OPTIONS-000091
I20241122 14:10:06.495311 1735 rsync_server.cc:149] rsync snapshot file: 0/MANIFEST-000089
I20241122 14:10:06.495317 1735 rsync_server.cc:149] rsync snapshot file: 0/000088.log
I20241122 14:10:06.495324 1735 rsync_server.cc:149] rsync snapshot file: 1/CURRENT
I20241122 14:10:06.495337 1735 rsync_server.cc:149] rsync snapshot file: 1/OPTIONS-000091
I20241122 14:10:06.495344 1735 rsync_server.cc:149] rsync snapshot file: 1/MANIFEST-000089
I20241122 14:10:06.495352 1735 rsync_server.cc:149] rsync snapshot file: 1/000088.log
I20241122 14:10:06.495360 1735 rsync_server.cc:149] rsync snapshot file: 2/CURRENT
I20241122 14:10:06.495368 1735 rsync_server.cc:149] rsync snapshot file: 2/OPTIONS-000091
I20241122 14:10:06.495376 1735 rsync_server.cc:149] rsync snapshot file: 2/MANIFEST-000089
I20241122 14:10:06.495399 1735 rsync_server.cc:149] rsync snapshot file: 2/000088.log
I20241122 14:10:06.495409 1735 rsync_server.cc:149] rsync snapshot file: info
I20241122 14:10:06.993593 1712 pika_repl_server_conn.cc:114] Receive Trysync, Slave ip: 10.42.128.192, Slave port:9221, DB: db0, filenum: 29, pro_offset: 95216551
I20241122 14:10:06.998489 1712 pika_repl_server_conn.cc:167] DB: db0 TrySync Success, Session: 2
I20241122 14:10:15.776841 1710 pika_repl_server_thread.cc:24] ServerThread Close Slave Conn, fd: 79, ip_port: 10.42.232.105:48878
I20241122 14:10:15.776893 1710 pika_server.cc:544] Delete Slave Success, ip_port: 10.42.232.105:9221
I20241122 14:10:15.776940 1710 pika_rm.cc:80] Remove Slave Node, DB: (db1), ip_port: 10.42.232.105:9221
I20241122 14:10:15.776968 1710 pika_rm.cc:80] Remove Slave Node, DB: (db0), ip_port: 10.42.232.105:9221
W20241122 14:11:09.535969 1734 rsync_server.cc:233] ip_port: 10.42.128.192:53694 connection closed

slave

path : /pika/conf/pika.conf
-----------Pika server----------
pika_version: 4.0.0
pika_git_sha:23d9bf6f4a400f3648be514d5f674bd9b8ae632f
pika_build_compile_date: 2024-09-27 03:02:24
redis_version: 4.0.0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20241122 14:10:05.994488 1 base_conf.cc:336] 1 port 9221
I20241122 14:10:05.997673 1 base_conf.cc:336] 2 db-instance-num 3
I20241122 14:10:05.997687 1 base_conf.cc:336] 3 rocksdb-ttl-second 86400 * 7;
I20241122 14:10:05.997696 1 base_conf.cc:336] 4 rocksdb-periodic-second 86400 * 3;
I20241122 14:10:05.997706 1 base_conf.cc:336] 5 thread-num 1
I20241122 14:10:05.997715 1 base_conf.cc:336] 6 rtc-cache-read yes
I20241122 14:10:05.997725 1 base_conf.cc:336] 7 thread-pool-size 12
I20241122 14:10:05.997733 1 base_conf.cc:336] 8 slow-cmd-pool no
I20241122 14:10:05.997742 1 base_conf.cc:336] 9 slow-cmd-thread-pool-size 1
I20241122 14:10:05.997751 1 base_conf.cc:336] 10 admin-thread-pool-size 2
I20241122 14:10:05.997761 1 base_conf.cc:336] 11 slow-cmd-list
I20241122 14:10:05.997771 1 base_conf.cc:336] 12 admin-cmd-list info, ping, monitor
I20241122 14:10:05.997779 1 base_conf.cc:336] 13 sync-thread-num 6
I20241122 14:10:05.997788 1 base_conf.cc:336] 14 sync-binlog-thread-num 1
I20241122 14:10:05.997797 1 base_conf.cc:336] 15 log-path ./data/log/
I20241122 14:10:05.997807 1 base_conf.cc:336] 16 log-retention-time 7
I20241122 14:10:05.997815 1 base_conf.cc:336] 17 db-path ./data/db/
I20241122 14:10:05.997824 1 base_conf.cc:336] 18 write-buffer-size 256M
I20241122 14:10:05.997834 1 base_conf.cc:336] 19 arena-block-size
I20241122 14:10:05.997843 1 base_conf.cc:336] 20 timeout 60
I20241122 14:10:05.997853 1 base_conf.cc:336] 21 requirepass
I20241122 14:10:05.997861 1 base_conf.cc:336] 22 masterauth
I20241122 14:10:05.997870 1 base_conf.cc:336] 23 userpass wenhuag6
I20241122 14:10:05.997879 1 base_conf.cc:336] 24 instance-mode classic
I20241122 14:10:05.997888 1 base_conf.cc:336] 25 databases 2
I20241122 14:10:05.997897 1 base_conf.cc:336] 26 replication-num 0
I20241122 14:10:05.997906 1 base_conf.cc:336] 27 consensus-level 0
I20241122 14:10:05.997915 1 base_conf.cc:336] 28 dump-prefix
I20241122 14:10:05.997924 1 base_conf.cc:336] 29 dump-path ./data/dump/
I20241122 14:10:05.997933 1 base_conf.cc:336] 30 dump-expire 0
I20241122 14:10:05.997943 1 base_conf.cc:336] 31 pidfile ./pika.pid
I20241122 14:10:05.997952 1 base_conf.cc:336] 32 maxclients 20000
I20241122 14:10:05.997961 1 base_conf.cc:336] 33 target-file-size-base 20M
I20241122 14:10:05.997970 1 base_conf.cc:336] 34 expire-logs-days 7
I20241122 14:10:05.997979 1 base_conf.cc:336] 35 expire-logs-nums 10
I20241122 14:10:05.997988 1 base_conf.cc:336] 36 root-connection-num 2
I20241122 14:10:05.997997 1 base_conf.cc:336] 37 slowlog-write-errorlog yes
I20241122 14:10:05.998018 1 base_conf.cc:336] 38 slowlog-log-slower-than 10000
I20241122 14:10:05.998023 1 base_conf.cc:336] 39 slowlog-max-len 128
I20241122 14:10:05.998028 1 base_conf.cc:336] 40 db-sync-path ./data/dbsync/
I20241122 14:10:05.998032 1 base_conf.cc:336] 41 db-sync-speed -1
I20241122 14:10:05.998037 1 base_conf.cc:336] 42 slave-priority 100
I20241122 14:10:05.998042 1 base_conf.cc:336] 43 disable_auto_compactions false
I20241122 14:10:05.998046 1 base_conf.cc:336] 44 max-subcompactions 1
I20241122 14:10:05.998051 1 base_conf.cc:336] 45 sync-window-size 9000
I20241122 14:10:05.998056 1 base_conf.cc:336] 46 max-conn-rbuf-size 268435456
I20241122 14:10:05.998061 1 base_conf.cc:336] 47 write-binlog yes
I20241122 14:10:05.998065 1 base_conf.cc:336] 48 binlog-file-size 104857600
I20241122 14:10:05.998070 1 base_conf.cc:336] 49 max-cache-statistic-keys 0
I20241122 14:10:05.998075 1 base_conf.cc:336] 50 small-compaction-threshold 5000
I20241122 14:10:05.998080 1 base_conf.cc:336] 51 small-compaction-duration-threshold 10000
I20241122 14:10:05.998085 1 base_conf.cc:336] 52 max-write-buffer-size 10737418240
I20241122 14:10:05.998090 1 base_conf.cc:336] 53 max-write-buffer-num 2
I20241122 14:10:05.998095 1 base_conf.cc:336] 54 min-write-buffer-number-to-merge 1
I20241122 14:10:05.998099 1 base_conf.cc:336] 55 max-total-wal-size 1073741824
I20241122 14:10:05.998103 1 base_conf.cc:336] 56 level0-stop-writes-trigger 36
I20241122 14:10:05.998108 1 base_conf.cc:336] 57 level0-slowdown-writes-trigger 20
I20241122 14:10:05.998113 1 base_conf.cc:336] 58 level0-file-num-compaction-trigger 4
I20241122 14:10:05.998117 1 base_conf.cc:336] 59 enable-db-statistics no
I20241122 14:10:05.998122 1 base_conf.cc:336] 60 db-statistics-level 2
I20241122 14:10:05.998127 1 base_conf.cc:336] 61 max-client-response-size 1073741824
I20241122 14:10:05.998131 1 base_conf.cc:336] 62 compression snappy
I20241122 14:10:05.998136 1 base_conf.cc:336] 63 max-background-jobs 3
I20241122 14:10:05.998142 1 base_conf.cc:336] 64 max-background-flushes -1
I20241122 14:10:05.998147 1 base_conf.cc:336] 65 max-background-compactions -1
I20241122 14:10:05.998150 1 base_conf.cc:336] 66 delayed-write-rate 0
I20241122 14:10:05.998155 1 base_conf.cc:336] 67 max-compaction-bytes -1
I20241122 14:10:05.998160 1 base_conf.cc:336] 68 max-cache-files 5000
I20241122 14:10:05.998165 1 base_conf.cc:336] 69 max-bytes-for-level-multiplier 10
I20241122 14:10:05.998169 1 base_conf.cc:336] 70 slotmigrate no
I20241122 14:10:05.998174 1 base_conf.cc:336] 71 slotmigrate-thread-num 1
I20241122 14:10:05.998179 1 base_conf.cc:336] 72 thread-migrate-keys-num 64
I20241122 14:10:05.998184 1 base_conf.cc:336] 73 default-slot-num 1024
I20241122 14:10:05.998189 1 base_conf.cc:336] 74 throttle-bytes-per-second 207200000
I20241122 14:10:05.998194 1 base_conf.cc:336] 75 rsync-timeout-ms 1000
I20241122 14:10:05.998199 1 base_conf.cc:336] 76 max-rsync-parallel-num 4
I20241122 14:10:05.998204 1 base_conf.cc:336] 77 cache-num 16
I20241122 14:10:05.998209 1 base_conf.cc:336] 78 cache-model 1
I20241122 14:10:05.998214 1 base_conf.cc:336] 79 cache-type string, set, zset, list, hash, bit
I20241122 14:10:05.998219 1 base_conf.cc:336] 80 zset-cache-field-num-per-key 512
I20241122 14:10:05.998224 1 base_conf.cc:336] 81 zset-cache-start-direction 0
I20241122 14:10:05.998229 1 base_conf.cc:336] 82 cache-maxmemory 10737418240
I20241122 14:10:05.998234 1 base_conf.cc:336] 83 cache-maxmemory-policy 1
I20241122 14:10:05.998239 1 base_conf.cc:336] 84 cache-maxmemory-samples 5
I20241122 14:10:05.998243 1 base_conf.cc:336] 85 cache-lfu-decay-time 1
I20241122 14:10:05.998248 1 base_conf.cc:336] 86 internal-used-unfinished-full-sync
I20241122 14:10:05.998253 1 base_conf.cc:336] 87 wash-data true
I20241122 14:10:05.998258 1 base_conf.cc:336] 88 replication-id k8shbwh
I20241122 14:10:05.998262 1 base_conf.cc:336] 89 slaveof 10.43.138.52:9221
I20241122 14:10:05.998440 1 pika.cc:209] Server at: /pika/conf/pika.conf
I20241122 14:10:05.999073 1 net_interfaces.cc:108] Using Networker Interface: eth0
I20241122 14:10:05.999284 1 net_interfaces.cc:152] got ip 10.42.128.192
I20241122 14:10:05.999315 1 pika_server.cc:157] host: 10.42.128.192 port: 9221
I20241122 14:10:05.999343 1 pika_server.cc:71] Worker queue limit is 20100
W20241122 14:10:05.999361 1 pika_server.cc:72] 0.0.0.0
I20241122 14:10:06.001734 1 pika_server.cc:1664] Dump file is not exist,path: ./data/dump/
I20241122 14:10:06.002171 1 pika_binlog.cc:81] Binlog: Manifest file not exist, we create a new one.
I20241122 14:10:06.002581 1 pika_binlog.cc:81] Binlog: Manifest file not exist, we create a new one.
I20241122 14:10:06.100045 1 pika_db.cc:50] db0 DB Success
I20241122 14:10:06.100252 1114 pika_cache_load_thread.cc:181] PikaCacheLoadThread::ThreadMain Start
I20241122 14:10:06.185447 1 pika_db.cc:50] db1 DB Success
I20241122 14:10:06.185635 1161 pika_cache_load_thread.cc:181] PikaCacheLoadThread::ThreadMain Start
I20241122 14:10:06.191804 1 net_util.cc:121] TimerTaskThread Starting...
I20241122 14:10:06.192011 1 pika_server.cc:214] Pika Server going to start
I20241122 14:10:06.192027 1 rsync_server.cc:48] start RsyncServer ...
I20241122 14:10:06.192274 1 rsync_server.cc:60] RsyncServer started ...
I20241122 14:10:06.192724 1193 pika_repl_client.cc:199] Try Send Meta Sync Request to Master (10.43.138.52:9221)
I20241122 14:10:06.194434 1163 pika_server.cc:452] Mark try connect finish
I20241122 14:10:06.194459 1163 pika_repl_client_conn.cc:155] Finish to handle meta sync response
I20241122 14:10:06.337046 1163 pika_repl_client_conn.cc:234] DB: db0 Need To Try DBSync
I20241122 14:10:06.337152 1163 pika_repl_client_conn.cc:231] DB: db1 TrySync Ok
I20241122 14:10:06.394372 1163 pika_repl_client_conn.cc:186] DB: db0 Need Wait To Sync
I20241122 14:10:06.394709 1163 base_conf.cc:345] ret OK
W20241122 14:10:06.398481 1163 env.cc:156] filesystem error: cannot rename: Device or resource busy [/pika/conf/pika.conf.tmp] [/pika/conf/pika.conf]
W20241122 14:10:06.493536 1193 pika_rm.cc:510] Slave DB: db0 Activating Rsync ... (retry count:0)
I20241122 14:10:06.495365 1193 rsync_client.cc:387] receive rsync meta infos, snapshot_uuid: d1ddbf4abaa8e8b966c12047f486c403files count: 13
W20241122 14:10:06.495438 1193 rsync_client.cc:403] DUMP_META_DATA not exist
I20241122 14:10:06.496219 1193 rsync_client.cc:336] copy meta data done, db name: db0 snapshot_uuid: d1ddbf4abaa8e8b966c12047f486c403 file count: 13 expired file count: 0 local file count: 0 remote file count: 13 remote snapshot_uuid: d1ddbf4abaa8e8b966c12047f486c403 local snapshot_uuid: file_set_: 13
W20241122 14:10:06.496253 1193 rsync_client.cc:346] file_set: 0/000088.log
W20241122 14:10:06.496276 1193 rsync_client.cc:346] file_set: 0/CURRENT
W20241122 14:10:06.496299 1193 rsync_client.cc:346] file_set: 0/MANIFEST-000089
W20241122 14:10:06.496321 1193 rsync_client.cc:346] file_set: 0/OPTIONS-000091
W20241122 14:10:06.496343 1193 rsync_client.cc:346] file_set: 1/000088.log
W20241122 14:10:06.496366 1193 rsync_client.cc:346] file_set: 1/CURRENT
W20241122 14:10:06.496388 1193 rsync_client.cc:346] file_set: 1/MANIFEST-000089
W20241122 14:10:06.496410 1193 rsync_client.cc:346] file_set: 1/OPTIONS-000091
W20241122 14:10:06.496433 1193 rsync_client.cc:346] file_set: 2/000088.log
W20241122 14:10:06.496455 1193 rsync_client.cc:346] file_set: 2/CURRENT
W20241122 14:10:06.496479 1193 rsync_client.cc:346] file_set: 2/MANIFEST-000089
W20241122 14:10:06.496500 1193 rsync_client.cc:346] file_set: 2/OPTIONS-000091
W20241122 14:10:06.496523 1193 rsync_client.cc:346] file_set: info
I20241122 14:10:06.496546 1193 rsync_client.cc:76] RsyncClient recover success
I20241122 14:10:06.496685 1199 rsync_client.cc:90] RsyncClient begin to copy remote files
I20241122 14:10:06.497074 1200 rsync_client.cc:40] copy remote file, filename: 0/000088.log
I20241122 14:10:06.497152 1201 rsync_client.cc:40] copy remote file, filename: 0/CURRENT
I20241122 14:10:06.497327 1202 rsync_client.cc:40] copy remote file, filename: 0/MANIFEST-000089
I20241122 14:10:06.497414 1203 rsync_client.cc:40] copy remote file, filename: 0/OPTIONS-000091
I20241122 14:10:06.556617 1200 rsync_client.cc:40] copy remote file, filename: 1/000088.log
I20241122 14:10:06.561188 1201 rsync_client.cc:40] copy remote file, filename: 1/CURRENT
I20241122 14:10:06.575455 1202 rsync_client.cc:40] copy remote file, filename: 1/MANIFEST-000089
I20241122 14:10:06.577225 1200 rsync_client.cc:40] copy remote file, filename: 2/000088.log
I20241122 14:10:06.580255 1201 rsync_client.cc:40] copy remote file, filename: 2/CURRENT
I20241122 14:10:06.582846 1202 rsync_client.cc:40] copy remote file, filename: 2/MANIFEST-000089
I20241122 14:10:06.583817 1203 rsync_client.cc:40] copy remote file, filename: 1/OPTIONS-000091
I20241122 14:10:06.585206 1200 rsync_client.cc:40] copy remote file, filename: info
I20241122 14:10:06.585965 1201 rsync_client.cc:53] work_thread index: 1 copy remote files done
I20241122 14:10:06.587874 1202 rsync_client.cc:53] work_thread index: 2 copy remote files done
I20241122 14:10:06.589839 1200 rsync_client.cc:53] work_thread index: 0 copy remote files done
I20241122 14:10:06.592157 1203 rsync_client.cc:40] copy remote file, filename: 2/OPTIONS-000091
I20241122 14:10:06.642392 1203 rsync_client.cc:53] work_thread index: 3 copy remote files done
I20241122 14:10:06.642485 1199 rsync_client.cc:153] RsyncClient copy remote files done
I20241122 14:10:06.696882 1193 pika_db.cc:500] DB: db0 Information from dbsync info, master_ip: 10.42.232.116, master_port: 9221, filenum: 29, offset: 95216551, term: 0, index: 0
I20241122 14:10:06.696993 1193 pika_db.cc:555] DB: db0, Prepare change db from: ./data/db/db0_bak
I20241122 14:10:06.742534 1193 pika_db.cc:576] DB: db0, Change db success
I20241122 14:10:06.851644 1193 base_conf.cc:345] ret OK
W20241122 14:10:06.851837 1193 env.cc:156] filesystem error: cannot rename: Device or resource busy [/pika/conf/pika.conf.tmp] [/pika/conf/pika.conf]
I20241122 14:10:06.998688 1163 pika_repl_client_conn.cc:231] DB: db0 TrySync Ok
I20241122 14:11:09.535208 1198 client_thread.cc:224] Do cron task del fd 113

@loverkoo
Copy link
Author

从日志上来看，master下没有sst文件。如果能复现的话，麻烦贴下master和slave端的日志，db目录下sst文件个数，dump目录下sst文件个数。

db & dump
master

root@pika401-5f49689565-ks6jh:/pika# ls -lhR data/db
data/db:
total 8.0K
drwxr-xr-x 11 root root 4.0K Nov 13 14:52 db0
drwxr-xr-x 11 root root 4.0K Nov 13 14:52 db1

data/db/db0:
total 36K
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 0
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 1
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 2
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 hashes
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 lists
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 sets
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 streams
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 strings
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 zsets

data/db/db0/0:
total 287M
-rw-r--r-- 1 root root 19 Nov 22 14:08 000088.log
-rw-r--r-- 1 root root 16 Nov 22 14:08 CURRENT
-rw-r--r-- 1 root root 36 Nov 12 16:51 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:51 LOCK
-rw-r--r-- 1 root root 110K Nov 22 14:18 LOG
-rw-r--r-- 1 root root 133K Nov 12 16:56 LOG.old.1731401759625411
-rw-r--r-- 1 root root 8.4K Nov 12 16:55 LOG.old.1731401760772568
-rw-r--r-- 1 root root 8.4K Nov 12 16:56 LOG.old.1731401778018483
-rw-r--r-- 1 root root 8.4K Nov 12 16:56 LOG.old.1731401805081994
-rw-r--r-- 1 root root 529K Nov 13 14:52 LOG.old.1731481264317766
-rw-r--r-- 1 root root 171K Nov 13 15:59 LOG.old.1731546528646444
-rw-r--r-- 1 root root 133K Nov 14 09:09 LOG.old.1732255718817860
-rw-r--r-- 1 root root 650 Nov 22 14:08 MANIFEST-000089
-rw-r--r-- 1 root root 35K Nov 14 09:09 OPTIONS-000087
-rw-r--r-- 1 root root 35K Nov 22 14:08 OPTIONS-000091

data/db/db0/1:
total 287M
-rw-r--r-- 1 root root 19 Nov 22 14:08 000088.log
-rw-r--r-- 1 root root 16 Nov 22 14:08 CURRENT
-rw-r--r-- 1 root root 36 Nov 12 16:51 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:51 LOCK
-rw-r--r-- 1 root root 110K Nov 22 14:18 LOG
-rw-r--r-- 1 root root 133K Nov 12 16:56 LOG.old.1731401805160436
-rw-r--r-- 1 root root 529K Nov 13 14:52 LOG.old.1731481264351003
-rw-r--r-- 1 root root 171K Nov 13 15:59 LOG.old.1731546528694224
-rw-r--r-- 1 root root 133K Nov 14 09:09 LOG.old.1732255718885266
-rw-r--r-- 1 root root 650 Nov 22 14:08 MANIFEST-000089
-rw-r--r-- 1 root root 35K Nov 14 09:09 OPTIONS-000087
-rw-r--r-- 1 root root 35K Nov 22 14:08 OPTIONS-000091

data/db/db0/2:
total 287M
-rw-r--r-- 1 root root 19 Nov 22 14:08 000088.log
-rw-r--r-- 1 root root 16 Nov 22 14:08 CURRENT
-rw-r--r-- 1 root root 36 Nov 12 16:51 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:51 LOCK
-rw-r--r-- 1 root root 110K Nov 22 14:18 LOG
-rw-r--r-- 1 root root 133K Nov 12 16:56 LOG.old.1731401805235485
-rw-r--r-- 1 root root 529K Nov 13 14:52 LOG.old.1731481264382579
-rw-r--r-- 1 root root 171K Nov 13 15:59 LOG.old.1731546528743879
-rw-r--r-- 1 root root 133K Nov 14 09:09 LOG.old.1732255718960887
-rw-r--r-- 1 root root 650 Nov 22 14:08 MANIFEST-000089
-rw-r--r-- 1 root root 35K Nov 14 09:09 OPTIONS-000087
-rw-r--r-- 1 root root 35K Nov 22 14:08 OPTIONS-000091

data/db/db0/hashes:
total 1.3G
-rw-r--r-- 2 root root 21M Nov 12 16:49 000206.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000208.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000209.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000210.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000211.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000212.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000213.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000214.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000215.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000216.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000217.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000218.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000219.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000221.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000222.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000223.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000224.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000285.sst
-rw-r--r-- 2 root root 11M Nov 12 16:49 000286.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000287.sst
-rw-r--r-- 2 root root 21M Nov 12 16:49 000293.sst
-rw-r--r-- 2 root root 16M Nov 12 16:49 000294.sst
-rw-r--r-- 2 root root 3.2M Nov 12 16:49 000297.sst
-rw-r--r-- 2 root root 15K Nov 12 16:49 000300.sst
-rw-r--r-- 2 root root 2.9K Nov 13 14:52 000306.sst
-rw-r--r-- 2 root root 1.3K Nov 13 14:52 000311.sst
-rw-r--r-- 2 root root 21M Nov 13 14:52 000312.sst
-rw-r--r-- 2 root root 21M Nov 13 14:52 000313.sst
-rw-r--r-- 2 root root 16M Nov 13 16:46 000331.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000332.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000333.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000334.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000335.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000336.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000337.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000338.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000339.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000340.sst
-rw-r--r-- 2 root root 18M Nov 13 16:46 000341.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000342.sst
-rw-r--r-- 2 root root 20M Nov 13 16:46 000343.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000344.sst
-rw-r--r-- 2 root root 11K Nov 13 16:46 000345.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000346.sst
-rw-r--r-- 2 root root 16M Nov 13 16:46 000347.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000348.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000349.sst
-rw-r--r-- 2 root root 16M Nov 13 16:46 000350.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000351.sst
-rw-r--r-- 2 root root 16M Nov 13 16:46 000352.sst
-rw-r--r-- 2 root root 12M Nov 13 16:46 000353.sst
-rw-r--r-- 2 root root 20M Nov 13 16:46 000354.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000355.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000356.sst
-rw-r--r-- 2 root root 20M Nov 13 16:46 000357.sst
-rw-r--r-- 2 root root 20M Nov 13 16:46 000358.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000359.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000360.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000361.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000362.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000363.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000364.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000365.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000366.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000367.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000368.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000369.sst
-rw-r--r-- 2 root root 21M Nov 13 16:46 000370.sst
-rw-r--r-- 1 root root 0 Nov 22 13:18 000403.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 37 Nov 12 16:49 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:49 LOCK
-rw-r--r-- 1 root root 68K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 18K Nov 13 16:47 LOG.old.1731487676116174
-rw-r--r-- 1 root root 195K Nov 14 09:06 LOG.old.1731546560642985
-rw-r--r-- 1 root root 18K Nov 14 09:09 LOG.old.1731546560648418
-rw-r--r-- 1 root root 1.4M Nov 20 16:09 LOG.old.1732090479455204
-rw-r--r-- 1 root root 18K Nov 20 16:14 LOG.old.1732090479474897
-rw-r--r-- 1 root root 438K Nov 22 13:17 LOG.old.1732252652350812
-rw-r--r-- 1 root root 18K Nov 22 13:17 LOG.old.1732252652357755
-rw-r--r-- 1 root root 49K Nov 22 13:18 LOG.old.1732252689768116
-rw-r--r-- 1 root root 18K Nov 22 13:18 LOG.old.1732252689771192
-rw-r--r-- 1 root root 9.8K Nov 22 14:08 MANIFEST-000404
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000408
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000410

data/db/db0/lists:
total 2.1M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000077.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 37 Nov 12 16:50 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:50 LOCK
-rw-r--r-- 1 root root 60K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 18K Nov 13 16:47 LOG.old.1731487676186008
-rw-r--r-- 1 root root 179K Nov 14 09:06 LOG.old.1731546560706109
-rw-r--r-- 1 root root 18K Nov 14 09:09 LOG.old.1731546560707457
-rw-r--r-- 1 root root 1.3M Nov 20 16:09 LOG.old.1732090479974588
-rw-r--r-- 1 root root 18K Nov 20 16:14 LOG.old.1732090479989532
-rw-r--r-- 1 root root 408K Nov 22 13:17 LOG.old.1732252652537295
-rw-r--r-- 1 root root 18K Nov 22 13:17 LOG.old.1732252652541810
-rw-r--r-- 1 root root 46K Nov 22 13:18 LOG.old.1732252689901791
-rw-r--r-- 1 root root 18K Nov 22 13:18 LOG.old.1732252689904649
-rw-r--r-- 1 root root 226 Nov 22 14:08 MANIFEST-000078
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000082
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000084

data/db/db0/sets:
total 2.1M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000077.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 37 Nov 12 16:49 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:49 LOCK
-rw-r--r-- 1 root root 60K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 18K Nov 13 16:47 LOG.old.1731487676163616
-rw-r--r-- 1 root root 179K Nov 14 09:06 LOG.old.1731546560676883
-rw-r--r-- 1 root root 18K Nov 14 09:09 LOG.old.1731546560678198
-rw-r--r-- 1 root root 1.3M Nov 20 16:09 LOG.old.1732090479828536
-rw-r--r-- 1 root root 18K Nov 20 16:14 LOG.old.1732090479844597
-rw-r--r-- 1 root root 408K Nov 22 13:17 LOG.old.1732252652459402
-rw-r--r-- 1 root root 18K Nov 22 13:17 LOG.old.1732252652465570
-rw-r--r-- 1 root root 46K Nov 22 13:18 LOG.old.1732252689844175
-rw-r--r-- 1 root root 18K Nov 22 13:18 LOG.old.1732252689846962
-rw-r--r-- 1 root root 221 Nov 22 14:08 MANIFEST-000078
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000082
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000084

data/db/db0/streams:
total 2.1M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000062.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 36 Nov 13 14:52 IDENTITY
-rw-r--r-- 1 root root 0 Nov 13 14:52 LOCK
-rw-r--r-- 1 root root 59K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 18K Nov 13 16:47 LOG.old.1731487676230506
-rw-r--r-- 1 root root 179K Nov 14 09:06 LOG.old.1731546560759958
-rw-r--r-- 1 root root 18K Nov 14 09:09 LOG.old.1731546560761196
-rw-r--r-- 1 root root 1.3M Nov 20 16:09 LOG.old.1732090480320017
-rw-r--r-- 1 root root 18K Nov 20 16:14 LOG.old.1732090480333549
-rw-r--r-- 1 root root 408K Nov 22 13:17 LOG.old.1732252652722879
-rw-r--r-- 1 root root 18K Nov 22 13:17 LOG.old.1732252652730910
-rw-r--r-- 1 root root 46K Nov 22 13:18 LOG.old.1732252690047918
-rw-r--r-- 1 root root 18K Nov 22 13:18 LOG.old.1732252690050526
-rw-r--r-- 1 root root 219 Nov 22 14:08 MANIFEST-000063
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000067
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000069

data/db/db0/strings:
total 1.6M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000059.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 37 Nov 12 16:49 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:49 LOCK
-rw-r--r-- 1 root root 37K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 8.4K Nov 13 16:47 LOG.old.1731487676071359
-rw-r--r-- 1 root root 131K Nov 14 09:06 LOG.old.1731546560615746
-rw-r--r-- 1 root root 957K Nov 20 16:09 LOG.old.1732090479325598
-rw-r--r-- 1 root root 308K Nov 22 13:17 LOG.old.1732252634331764
-rw-r--r-- 1 root root 8.4K Nov 22 13:17 LOG.old.1732252636115253
-rw-r--r-- 1 root root 8.4K Nov 22 13:17 LOG.old.1732252652287547
-rw-r--r-- 1 root root 28K Nov 22 13:18 LOG.old.1732252670668176
-rw-r--r-- 1 root root 8.4K Nov 22 13:17 LOG.old.1732252672272379
-rw-r--r-- 1 root root 8.4K Nov 22 13:17 LOG.old.1732252689728213
-rw-r--r-- 1 root root 119 Nov 22 14:08 MANIFEST-000060
-rw-r--r-- 1 root root 7.2K Nov 22 13:18 OPTIONS-000062
-rw-r--r-- 1 root root 7.2K Nov 22 14:08 OPTIONS-000064

data/db/db0/zsets:
total 2.6M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000091.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 37 Nov 12 16:50 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:50 LOCK
-rw-r--r-- 1 root root 82K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 18K Nov 13 16:47 LOG.old.1731487676207835
-rw-r--r-- 1 root root 227K Nov 14 09:06 LOG.old.1731546560733118
-rw-r--r-- 1 root root 18K Nov 14 09:09 LOG.old.1731546560734595
-rw-r--r-- 1 root root 1.5M Nov 20 16:09 LOG.old.1732090480140624
-rw-r--r-- 1 root root 18K Nov 20 16:14 LOG.old.1732090480155706
-rw-r--r-- 1 root root 508K Nov 22 13:17 LOG.old.1732252652614598
-rw-r--r-- 1 root root 18K Nov 22 13:17 LOG.old.1732252652620885
-rw-r--r-- 1 root root 63K Nov 22 13:18 LOG.old.1732252689960566
-rw-r--r-- 1 root root 18K Nov 22 13:18 LOG.old.1732252689963213
-rw-r--r-- 1 root root 325 Nov 22 14:08 MANIFEST-000092
-rw-r--r-- 1 root root 17K Nov 22 14:08 OPTIONS-000098
-rw-r--r-- 1 root root 17K Nov 22 14:08 OPTIONS-000100

data/db/db1:
total 36K
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 0
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 1
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 2
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 hashes
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 lists
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 sets
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 streams
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 strings
drwxr-xr-x 2 root root 4.0K Nov 22 14:08 zsets

data/db/db1/0:
total 287M
-rw-r--r-- 1 root root 19 Nov 22 14:08 000088.log
-rw-r--r-- 1 root root 16 Nov 22 14:08 CURRENT
-rw-r--r-- 1 root root 36 Nov 12 16:51 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:51 LOCK
-rw-r--r-- 1 root root 110K Nov 22 14:18 LOG
-rw-r--r-- 1 root root 132K Nov 12 16:56 LOG.old.1731401805305641
-rw-r--r-- 1 root root 529K Nov 13 14:52 LOG.old.1731481264416775
-rw-r--r-- 1 root root 170K Nov 13 15:59 LOG.old.1731546528791693
-rw-r--r-- 1 root root 133K Nov 14 09:09 LOG.old.1732255719053113
-rw-r--r-- 1 root root 650 Nov 22 14:08 MANIFEST-000089
-rw-r--r-- 1 root root 35K Nov 14 09:09 OPTIONS-000087
-rw-r--r-- 1 root root 35K Nov 22 14:08 OPTIONS-000091

data/db/db1/1:
total 287M
-rw-r--r-- 1 root root 19 Nov 22 14:08 000088.log
-rw-r--r-- 1 root root 16 Nov 22 14:08 CURRENT
-rw-r--r-- 1 root root 36 Nov 12 16:51 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:51 LOCK
-rw-r--r-- 1 root root 110K Nov 22 14:18 LOG
-rw-r--r-- 1 root root 132K Nov 12 16:56 LOG.old.1731401805371985
-rw-r--r-- 1 root root 529K Nov 13 14:52 LOG.old.1731481264447767
-rw-r--r-- 1 root root 170K Nov 13 15:59 LOG.old.1731546528839608
-rw-r--r-- 1 root root 133K Nov 14 09:09 LOG.old.1732255719163250
-rw-r--r-- 1 root root 650 Nov 22 14:08 MANIFEST-000089
-rw-r--r-- 1 root root 35K Nov 14 09:09 OPTIONS-000087
-rw-r--r-- 1 root root 35K Nov 22 14:08 OPTIONS-000091

data/db/db1/2:
total 287M
-rw-r--r-- 1 root root 19 Nov 22 14:08 000088.log
-rw-r--r-- 1 root root 16 Nov 22 14:08 CURRENT
-rw-r--r-- 1 root root 36 Nov 12 16:51 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:51 LOCK
-rw-r--r-- 1 root root 110K Nov 22 14:18 LOG
-rw-r--r-- 1 root root 132K Nov 12 16:56 LOG.old.1731401805443838
-rw-r--r-- 1 root root 529K Nov 13 14:52 LOG.old.1731481264476919
-rw-r--r-- 1 root root 170K Nov 13 15:59 LOG.old.1731546528881048
-rw-r--r-- 1 root root 133K Nov 14 09:09 LOG.old.1732255719294709
-rw-r--r-- 1 root root 650 Nov 22 14:08 MANIFEST-000089
-rw-r--r-- 1 root root 35K Nov 14 09:09 OPTIONS-000087
-rw-r--r-- 1 root root 35K Nov 22 14:08 OPTIONS-000091

data/db/db1/hashes:
total 2.1M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000064.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 37 Nov 12 16:47 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:47 LOCK
-rw-r--r-- 1 root root 59K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 18K Nov 13 16:47 LOG.old.1731487676272226
-rw-r--r-- 1 root root 178K Nov 14 09:06 LOG.old.1731546560813490
-rw-r--r-- 1 root root 18K Nov 14 09:09 LOG.old.1731546560814841
-rw-r--r-- 1 root root 1.3M Nov 20 16:09 LOG.old.1732090480631005
-rw-r--r-- 1 root root 18K Nov 20 16:14 LOG.old.1732090480642839
-rw-r--r-- 1 root root 408K Nov 22 13:17 LOG.old.1732252652915972
-rw-r--r-- 1 root root 18K Nov 22 13:17 LOG.old.1732252652920951
-rw-r--r-- 1 root root 46K Nov 22 13:18 LOG.old.1732252690134775
-rw-r--r-- 1 root root 18K Nov 22 13:18 LOG.old.1732252690137700
-rw-r--r-- 1 root root 219 Nov 22 14:08 MANIFEST-000065
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000069
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000071

data/db/db1/lists:
total 2.1M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000064.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 37 Nov 12 16:47 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:47 LOCK
-rw-r--r-- 1 root root 59K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 18K Nov 13 16:47 LOG.old.1731487676315820
-rw-r--r-- 1 root root 178K Nov 14 09:06 LOG.old.1731546560870195
-rw-r--r-- 1 root root 18K Nov 14 09:09 LOG.old.1731546560871542
-rw-r--r-- 1 root root 1.3M Nov 20 16:09 LOG.old.1732090480994031
-rw-r--r-- 1 root root 18K Nov 20 16:14 LOG.old.1732090481009497
-rw-r--r-- 1 root root 408K Nov 22 13:17 LOG.old.1732252653097355
-rw-r--r-- 1 root root 18K Nov 22 13:17 LOG.old.1732252653101616
-rw-r--r-- 1 root root 46K Nov 22 13:18 LOG.old.1732252690262950
-rw-r--r-- 1 root root 18K Nov 22 13:18 LOG.old.1732252690265476
-rw-r--r-- 1 root root 226 Nov 22 14:08 MANIFEST-000065
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000069
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000071

data/db/db1/sets:
total 2.1M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000064.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 37 Nov 12 16:47 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:47 LOCK
-rw-r--r-- 1 root root 59K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 18K Nov 13 16:47 LOG.old.1731487676293949
-rw-r--r-- 1 root root 178K Nov 14 09:06 LOG.old.1731546560843121
-rw-r--r-- 1 root root 18K Nov 14 09:09 LOG.old.1731546560844527
-rw-r--r-- 1 root root 1.3M Nov 20 16:09 LOG.old.1732090480780649
-rw-r--r-- 1 root root 18K Nov 20 16:14 LOG.old.1732090480799196
-rw-r--r-- 1 root root 409K Nov 22 13:17 LOG.old.1732252653014207
-rw-r--r-- 1 root root 18K Nov 22 13:17 LOG.old.1732252653019976
-rw-r--r-- 1 root root 46K Nov 22 13:18 LOG.old.1732252690194607
-rw-r--r-- 1 root root 18K Nov 22 13:18 LOG.old.1732252690197140
-rw-r--r-- 1 root root 221 Nov 22 14:08 MANIFEST-000065
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000069
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000071

data/db/db1/streams:
total 2.1M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000062.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 36 Nov 13 14:52 IDENTITY
-rw-r--r-- 1 root root 0 Nov 13 14:52 LOCK
-rw-r--r-- 1 root root 59K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 18K Nov 13 16:47 LOG.old.1731487676363058
-rw-r--r-- 1 root root 178K Nov 14 09:06 LOG.old.1731546560929167
-rw-r--r-- 1 root root 18K Nov 14 09:09 LOG.old.1731546560930451
-rw-r--r-- 1 root root 1.3M Nov 20 16:09 LOG.old.1732090481488787
-rw-r--r-- 1 root root 18K Nov 20 16:14 LOG.old.1732090481500431
-rw-r--r-- 1 root root 408K Nov 22 13:17 LOG.old.1732252653286586
-rw-r--r-- 1 root root 18K Nov 22 13:17 LOG.old.1732252653290040
-rw-r--r-- 1 root root 40K Nov 22 13:18 LOG.old.1732252690431152
-rw-r--r-- 1 root root 18K Nov 22 13:18 LOG.old.1732252690433602
-rw-r--r-- 1 root root 219 Nov 22 14:08 MANIFEST-000063
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000067
-rw-r--r-- 1 root root 12K Nov 22 14:08 OPTIONS-000069

data/db/db1/strings:
total 1.6M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000046.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 37 Nov 12 16:47 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:47 LOCK
-rw-r--r-- 1 root root 37K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 15K Nov 12 16:51 LOG.old.1731480738018806
-rw-r--r-- 1 root root 28K Nov 13 15:01 LOG.old.1731484745093944
-rw-r--r-- 1 root root 37K Nov 13 16:42 LOG.old.1731487386241803
-rw-r--r-- 1 root root 28K Nov 13 16:47 LOG.old.1731487676251189
-rw-r--r-- 1 root root 130K Nov 14 09:06 LOG.old.1731546560787225
-rw-r--r-- 1 root root 957K Nov 20 16:09 LOG.old.1732090480470783
-rw-r--r-- 1 root root 308K Nov 22 13:17 LOG.old.1732252652835594
-rw-r--r-- 1 root root 28K Nov 22 13:18 LOG.old.1732252690097454
-rw-r--r-- 1 root root 119 Nov 22 14:08 MANIFEST-000047
-rw-r--r-- 1 root root 7.2K Nov 22 13:18 OPTIONS-000049
-rw-r--r-- 1 root root 7.2K Nov 22 14:08 OPTIONS-000051

data/db/db1/zsets:
total 2.6M
-rw-r--r-- 1 root root 0 Nov 22 13:18 000078.log
-rw-r--r-- 1 root root 16 Nov 22 13:18 CURRENT
-rw-r--r-- 1 root root 37 Nov 12 16:47 IDENTITY
-rw-r--r-- 1 root root 0 Nov 12 16:47 LOCK
-rw-r--r-- 1 root root 81K Nov 22 14:08 LOG
-rw-r--r-- 1 root root 18K Nov 13 16:47 LOG.old.1731487676337773
-rw-r--r-- 1 root root 226K Nov 14 09:06 LOG.old.1731546560898539
-rw-r--r-- 1 root root 18K Nov 14 09:09 LOG.old.1731546560899939
-rw-r--r-- 1 root root 1.5M Nov 20 16:09 LOG.old.1732090481228313
-rw-r--r-- 1 root root 18K Nov 20 16:14 LOG.old.1732090481242555
-rw-r--r-- 1 root root 508K Nov 22 13:17 LOG.old.1732252653176641
-rw-r--r-- 1 root root 18K Nov 22 13:17 LOG.old.1732252653180991
-rw-r--r-- 1 root root 55K Nov 22 13:18 LOG.old.1732252690331093
-rw-r--r-- 1 root root 18K Nov 22 13:18 LOG.old.1732252690333628
-rw-r--r-- 1 root root 325 Nov 22 14:08 MANIFEST-000079
-rw-r--r-- 1 root root 17K Nov 22 14:08 OPTIONS-000085
-rw-r--r-- 1 root root 17K Nov 22 14:08 OPTIONS-000087

root@pika401-5f49689565-ks6jh:/pika# ls -lhR data/dump
data/dump:
total 12K
drwxr-xr-x 3 root root 4.0K Nov 22 14:10 20241122

data/dump/20241122:
total 4.0K
drwxr-xr-x 5 root root 4.0K Nov 22 14:10 db0

data/dump/20241122/db0:
total 16K
drwxr-xr-x 2 root root 4.0K Nov 22 14:10 0
drwxr-xr-x 2 root root 4.0K Nov 22 14:10 1
drwxr-xr-x 2 root root 4.0K Nov 22 14:10 2
-rw-r--r-- 1 root root 34 Nov 22 14:10 info

data/dump/20241122/db0/0:
total 48K
-rw-r--r-- 1 root root 19 Nov 22 14:10 000088.log
-rw-r--r-- 1 root root 16 Nov 22 14:10 CURRENT
-rw-r--r-- 1 root root 650 Nov 22 14:10 MANIFEST-000089
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000091

data/dump/20241122/db0/1:
total 48K
-rw-r--r-- 1 root root 19 Nov 22 14:10 000088.log
-rw-r--r-- 1 root root 16 Nov 22 14:10 CURRENT
-rw-r--r-- 1 root root 650 Nov 22 14:10 MANIFEST-000089
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000091

data/dump/20241122/db0/2:
total 48K
-rw-r--r-- 1 root root 19 Nov 22 14:10 000088.log
-rw-r--r-- 1 root root 16 Nov 22 14:10 CURRENT
-rw-r--r-- 1 root root 650 Nov 22 14:10 MANIFEST-000089
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000091

slave

root@pika4-55cf7c8bc4-h2hw2:/pika# ls -lhR data/db
data/db:
total 8.0K
drwxr-xr-x 5 root root 4.0K Nov 22 14:10 db0
drwxr-xr-x 5 root root 4.0K Nov 22 14:10 db1

data/db/db0:
total 16K
drwxr-xr-x 2 root root 4.0K Nov 22 14:10 0
drwxr-xr-x 2 root root 4.0K Nov 22 14:10 1
drwxr-xr-x 2 root root 4.0K Nov 22 14:10 2
-rw-r--r-- 1 root root 192 Nov 22 14:10 DUMP_META_DATA

data/db/db0/0:
total 4.2M
-rw-r--r-- 1 root root 0 Nov 22 14:10 000092.log
-rw-r--r-- 1 root root 16 Nov 22 14:10 CURRENT
-rw-r--r-- 1 root root 36 Nov 22 14:10 IDENTITY
-rw-r--r-- 1 root root 0 Nov 22 14:10 LOCK
-rw-r--r-- 1 root root 112K Nov 22 14:20 LOG
-rw-r--r-- 1 root root 650 Nov 22 14:10 MANIFEST-000093
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000091
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000095

data/db/db0/1:
total 4.2M
-rw-r--r-- 1 root root 0 Nov 22 14:10 000092.log
-rw-r--r-- 1 root root 16 Nov 22 14:10 CURRENT
-rw-r--r-- 1 root root 36 Nov 22 14:10 IDENTITY
-rw-r--r-- 1 root root 0 Nov 22 14:10 LOCK
-rw-r--r-- 1 root root 112K Nov 22 14:20 LOG
-rw-r--r-- 1 root root 650 Nov 22 14:10 MANIFEST-000093
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000091
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000095

data/db/db0/2:
total 4.2M
-rw-r--r-- 1 root root 0 Nov 22 14:10 000092.log
-rw-r--r-- 1 root root 16 Nov 22 14:10 CURRENT
-rw-r--r-- 1 root root 36 Nov 22 14:10 IDENTITY
-rw-r--r-- 1 root root 0 Nov 22 14:10 LOCK
-rw-r--r-- 1 root root 112K Nov 22 14:20 LOG
-rw-r--r-- 1 root root 650 Nov 22 14:10 MANIFEST-000093
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000091
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000095

data/db/db1:
total 12K
drwxr-xr-x 2 root root 4.0K Nov 22 14:10 0
drwxr-xr-x 2 root root 4.0K Nov 22 14:10 1
drwxr-xr-x 2 root root 4.0K Nov 22 14:10 2

data/db/db1/0:
total 286M
-rw-r--r-- 1 root root 19 Nov 22 14:10 000004.log
-rw-r--r-- 1 root root 16 Nov 22 14:10 CURRENT
-rw-r--r-- 1 root root 36 Nov 22 14:10 IDENTITY
-rw-r--r-- 1 root root 0 Nov 22 14:10 LOCK
-rw-r--r-- 1 root root 109K Nov 22 14:20 LOG
-rw-r--r-- 1 root root 455 Nov 22 14:10 MANIFEST-000005
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000017
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000019

data/db/db1/1:
total 286M
-rw-r--r-- 1 root root 19 Nov 22 14:10 000004.log
-rw-r--r-- 1 root root 16 Nov 22 14:10 CURRENT
-rw-r--r-- 1 root root 36 Nov 22 14:10 IDENTITY
-rw-r--r-- 1 root root 0 Nov 22 14:10 LOCK
-rw-r--r-- 1 root root 109K Nov 22 14:20 LOG
-rw-r--r-- 1 root root 455 Nov 22 14:10 MANIFEST-000005
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000017
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000019

data/db/db1/2:
total 286M
-rw-r--r-- 1 root root 19 Nov 22 14:10 000004.log
-rw-r--r-- 1 root root 16 Nov 22 14:10 CURRENT
-rw-r--r-- 1 root root 36 Nov 22 14:10 IDENTITY
-rw-r--r-- 1 root root 0 Nov 22 14:10 LOCK
-rw-r--r-- 1 root root 109K Nov 22 14:20 LOG
-rw-r--r-- 1 root root 455 Nov 22 14:10 MANIFEST-000005
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000017
-rw-r--r-- 1 root root 35K Nov 22 14:10 OPTIONS-000019

@wangshao1
Copy link
Collaborator

db0/0, db0/1, db0/2以及db1/0, db1/1, db1/2下的数据是4.0版本的，类似db0/lists这些是3.x版本的，4.0和3.x版本数据不兼容。
master端4.0版本下没有sst文件，所以slave也没有sst数据。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.

The data under db0/0, db0/1, db0/2 and db1/0, db1/1, db1/2 are version 4.0, similar to db0/lists are version 3.x, 4.0 and 3.x version data Not compatible.
There is no sst file under the master version 4.0, so the slave does not have sst data.

@loverkoo
Copy link
Author

db0/0, db0/1, db0/2以及db1/0, db1/1, db1/2下的数据是4.0版本的，类似db0/lists这些是3.x版本的，4.0和3.x版本数据不兼容。 master端4.0版本下没有sst文件，所以slave也没有sst数据。

那3.x到4.0的升级方案是怎样的呢

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.

The data under db0/0, db0/1, db0/2 and db1/0, db1/1, db1/2 are version 4.0, similar to db0/lists are version 3.x, versions 4.0 and 3.x The data is incompatible. There is no sst file under the master version 4.0, so the slave does not have sst data.

So what is the upgrade plan from 3.x to 4.0?

@wangshao1
Copy link
Collaborator

暂时没有生产环境验证过的迁移方式，3.x版本建议先继续使用3.x，pika还在维护。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.

There is currently no migration method that has been verified in the production environment. It is recommended to continue using version 3.x for version 3.x. Pika is still under maintenance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
☢️ Bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@wangshao1 @Issues-translate-bot @loverkoo