2024/07/08-15:14:33.021930 139818746500672 DEBG dcompact_worker.cpp:2211: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00: fork to child time = 0.008781 sec 2024/07/08-15:14:33.022617 139818746500672 DEBG dcompact_worker.cpp:1011: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00 2024/07/08-15:14:33.027391 139818746500672 DEBG dcompact_worker.cpp:1031: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00 2024/07/08-15:14:33.028797 139818746500672 TRAC dcompact_worker.cpp:1081: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/07/08-15:14:33.029074 139818746500672 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/07/08-15:14:33.063289 139818746500672 [version_set.cc:6139] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-02442/att-00/MANIFEST-12625661 2024/07/08-15:14:33.122268 139818746500672 [version_set.cc:6182] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-02442/att-00/MANIFEST-12625661 succeeded,manifest_file_number is 12625661, next_file_number is 12627118, last_sequence is 147212955546, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 12627024 2024/07/08-15:14:33.122283 139818746500672 [version_set.cc:6197] Column family [default] (ID 0), log number is 0 2024/07/08-15:14:33.122442 139818746500672 [version_set.cc:5652] Creating manifest 12627118 2024/07/08-15:14:33.146963 139818746500672 DEBG dcompact_worker.cpp:1215: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00: bottommost_level: fake = 1, rpc = 0 2024/07/08-15:14:33.160240 139818746500672 [compaction_job.cc:2433] [default] [JOB 2442] Compacting 10@4 + 16@5 files to L5, score -1.00, subcompactions 1 : 1 2024/07/08-15:14:33.160269 139818746500672 [compaction_job.cc:2441] [default]: Compaction start summary: Base version 1775 Base level 4, inputs: [12625977(172MB) 12625978(172MB) 12625979(172MB) 12625980(172MB) 12625981(172MB) 12625982(172MB) 12625983(172MB) 12625984(172MB) 12625985(119MB) 12625986(172MB)], [12627115(330MB) 12627116(117MB) 12608620(43MB) 12625558(222MB) 12435388(94MB) 12583208(194MB) 12618546(268MB) 12509554(17MB) 12606818(258MB) 12606199(226MB) 12617089(168MB) 12574465(197MB) 12625559(253MB) 12603650(114MB) 12621854(189MB) 12609441(255MB)] 2024/07/08-15:14:33.163649 139818746500672 [top_zip_table.cc:59] core git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-15:14:33.163686 139818746500672 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-15:14:33.163713 139818746500672 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-15:14:33.163723 139818746500672 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 7f80399bfbd2071b6e0f5683e8ab7225b8e6e394 Author: leipeng Date: 2024-01-19 14:52:30 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/07/08-15:14:33.163743 139818746500672 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit 1240987ca13d0fe562c9b9c037155079f0877840 Author: leipeng Date: 2024-01-19 15:07:04 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/07/08-15:14:36.843832 139818746500672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627119.sst) /dev/shm/ToplingTemp/Topling-14573-JwwEU7 2024/07/08-15:14:40.383485 139818746500672 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627119.sst): first pass time = 3.54's, 151.676'MB/sec 2024/07/08-15:14:45.364832 139818746500672 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627119.sst): old prealloc_size = 590558003, real_size = 330796552 2024/07/08-15:14:45.364920 139818746500672 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627119.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.82 2024/07/08-15:14:47.515678 139818746500672 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627119.sst): second pass time = 4.98's, 99.192'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.15's, 160.920'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 10955.391'MB/sec (index lex order gen) rebuild zvType time = 1.79's, 0.374'MB/sec write SST data time = 0.36's, 965.642'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 96309.514'MB/sec zip my value time = 4.98's, unzip length = 0.494'GB zip my value throughput = 99.192'MB/sec zip pipeline throughput = 99.192'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.82 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4981 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 0.512712 GB avg = 0.191 KB (by entry num) total key len = 0.042950 GB avg = 0.016 KB total ukey len = 0.021475 GB avg = 0.008 KB total ukey num = 0.002684355 Billion total entry num = 0.002684355 Billion write speed all = 0.029086999 MB/sec (with seq num) write speed all = 0.027962864 MB/sec (without seq num) 2024/07/08-15:14:47.933382 139818746500672 [compaction_job.cc:1944] [default] [JOB 2442] Generated table #12627119: 2684355 keys, 346113483 bytes, temperature: kWarm 2024/07/08-15:14:47.933464 139818746500672 EVENT_LOG_v1 {"time_micros": 1720422887933418, "cf_name": "default", "job": 2442, "event": "table_file_creation", "file_number": 12627119, "file_size": 346113483, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142186179574, "largest_seqno": 147180687698, "table_properties": {"data_size": 329705632, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422876, "oldest_key_time": 0, "file_creation_time": 1720422876, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12627119, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:14:48.023653 139818746500672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627120.sst) /dev/shm/ToplingTemp/Topling-14573-fFVQy8 2024/07/08-15:14:54.181567 139818746500672 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627120.sst): first pass time = 6.16's, 87.185'MB/sec 2024/07/08-15:14:59.203983 139818746500672 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627120.sst): old prealloc_size = 590558003, real_size = 330800616 2024/07/08-15:14:59.204074 139818746500672 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627120.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:15:01.392399 139818746500672 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627120.sst): second pass time = 5.02's, 98.386'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.19's, 158.160'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 13056.836'MB/sec (index lex order gen) rebuild zvType time = 1.82's, 0.369'MB/sec write SST data time = 0.37's, 933.208'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 109441.069'MB/sec zip my value time = 5.02's, unzip length = 0.494'GB zip my value throughput = 98.386'MB/sec zip pipeline throughput = 98.386'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 1.025424 GB avg = 0.191 KB (by entry num) total key len = 0.085899 GB avg = 0.016 KB total ukey len = 0.042950 GB avg = 0.008 KB total ukey num = 0.005368710 Billion total entry num = 0.005368710 Billion write speed all = 0.058131771 MB/sec (with seq num) write speed all = 0.055885133 MB/sec (without seq num) 2024/07/08-15:15:01.791397 139818746500672 [compaction_job.cc:1944] [default] [JOB 2442] Generated table #12627120: 2684355 keys, 346117515 bytes, temperature: kWarm 2024/07/08-15:15:01.791470 139818746500672 EVENT_LOG_v1 {"time_micros": 1720422901791434, "cf_name": "default", "job": 2442, "event": "table_file_creation", "file_number": 12627120, "file_size": 346117515, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142140768186, "largest_seqno": 147180687703, "table_properties": {"data_size": 329709696, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422888, "oldest_key_time": 0, "file_creation_time": 1720422888, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12627120, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:15:01.904855 139818746500672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627121.sst) /dev/shm/ToplingTemp/Topling-14573-Sl0ZK6 2024/07/08-15:15:09.349380 139818746500672 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627121.sst): first pass time = 7.44's, 72.117'MB/sec 2024/07/08-15:15:14.518473 139818746500672 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627121.sst): old prealloc_size = 590558003, real_size = 330805160 2024/07/08-15:15:14.518555 139818746500672 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627121.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:15:16.707386 139818746500672 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627121.sst): second pass time = 5.17's, 95.594'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.19's, 158.125'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 12629.374'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.38's, 911.934'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 105752.679'MB/sec zip my value time = 5.17's, unzip length = 0.494'GB zip my value throughput = 95.594'MB/sec zip pipeline throughput = 95.594'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 1.538135 GB avg = 0.191 KB (by entry num) total key len = 0.128849 GB avg = 0.016 KB total ukey len = 0.064425 GB avg = 0.008 KB total ukey num = 0.008053065 Billion total entry num = 0.008053065 Billion write speed all = 0.087127858 MB/sec (with seq num) write speed all = 0.083760598 MB/sec (without seq num) 2024/07/08-15:15:17.084094 139818746500672 [compaction_job.cc:1944] [default] [JOB 2442] Generated table #12627121: 2684355 keys, 346122059 bytes, temperature: kWarm 2024/07/08-15:15:17.084162 139818746500672 EVENT_LOG_v1 {"time_micros": 1720422917084129, "cf_name": "default", "job": 2442, "event": "table_file_creation", "file_number": 12627121, "file_size": 346122059, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142293222012, "largest_seqno": 147180687706, "table_properties": {"data_size": 329714240, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422901, "oldest_key_time": 0, "file_creation_time": 1720422901, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12627121, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:15:17.190018 139818746500672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627122.sst) /dev/shm/ToplingTemp/Topling-14573-81jJo8 2024/07/08-15:15:27.661593 139818746500672 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627122.sst): first pass time = 10.47's, 51.270'MB/sec 2024/07/08-15:15:32.743474 139818746500672 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627122.sst): old prealloc_size = 590558003, real_size = 330798088 2024/07/08-15:15:32.743560 139818746500672 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627122.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:15:34.903560 139818746500672 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627122.sst): second pass time = 5.08's, 97.237'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.16's, 160.233'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 12982.122'MB/sec (index lex order gen) rebuild zvType time = 1.80's, 0.373'MB/sec write SST data time = 0.36's, 954.243'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 103470.215'MB/sec zip my value time = 5.08's, unzip length = 0.494'GB zip my value throughput = 97.237'MB/sec zip pipeline throughput = 97.237'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4981 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 2.050847 GB avg = 0.191 KB (by entry num) total key len = 0.171799 GB avg = 0.016 KB total ukey len = 0.085899 GB avg = 0.008 KB total ukey num = 0.010737420 Billion total entry num = 0.010737420 Billion write speed all = 0.116060098 MB/sec (with seq num) write speed all = 0.111574684 MB/sec (without seq num) 2024/07/08-15:15:35.276923 139818746500672 [compaction_job.cc:1944] [default] [JOB 2442] Generated table #12627122: 2684355 keys, 346115019 bytes, temperature: kWarm 2024/07/08-15:15:35.276995 139818746500672 EVENT_LOG_v1 {"time_micros": 1720422935276959, "cf_name": "default", "job": 2442, "event": "table_file_creation", "file_number": 12627122, "file_size": 346115019, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142170316361, "largest_seqno": 147180687705, "table_properties": {"data_size": 329707168, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422917, "oldest_key_time": 0, "file_creation_time": 1720422917, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12627122, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:15:35.380419 139818746500672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627123.sst) /dev/shm/ToplingTemp/Topling-14573-xa0F69 2024/07/08-15:15:42.375004 139818746500672 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627123.sst): first pass time = 6.99's, 76.756'MB/sec 2024/07/08-15:15:47.475600 139818746500672 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627123.sst): old prealloc_size = 590558003, real_size = 330784968 2024/07/08-15:15:47.475683 139818746500672 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627123.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.82 2024/07/08-15:15:49.658219 139818746500672 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627123.sst): second pass time = 5.10's, 96.871'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.18's, 158.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 13747.527'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.370'MB/sec write SST data time = 0.37's, 934.390'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 85183.649'MB/sec zip my value time = 5.10's, unzip length = 0.494'GB zip my value throughput = 96.871'MB/sec zip pipeline throughput = 96.871'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.82 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4981 dict = 1.00 all = 1.5512 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 2.563559 GB avg = 0.191 KB (by entry num) total key len = 0.214748 GB avg = 0.016 KB total ukey len = 0.107374 GB avg = 0.008 KB total ukey num = 0.013421775 Billion total entry num = 0.013421775 Billion write speed all = 0.144963436 MB/sec (with seq num) write speed all = 0.139360985 MB/sec (without seq num) 2024/07/08-15:15:50.038992 139818746500672 [compaction_job.cc:1944] [default] [JOB 2442] Generated table #12627123: 2684355 keys, 346101899 bytes, temperature: kWarm 2024/07/08-15:15:50.039062 139818746500672 EVENT_LOG_v1 {"time_micros": 1720422950039028, "cf_name": "default", "job": 2442, "event": "table_file_creation", "file_number": 12627123, "file_size": 346101899, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142481293885, "largest_seqno": 147180687709, "table_properties": {"data_size": 329694048, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422935, "oldest_key_time": 0, "file_creation_time": 1720422935, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12627123, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:15:50.152048 139818746500672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627124.sst) /dev/shm/ToplingTemp/Topling-14573-Fpbcc8 2024/07/08-15:15:59.124221 139818746500672 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627124.sst): first pass time = 8.97's, 59.838'MB/sec 2024/07/08-15:16:04.271809 139818746500672 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627124.sst): old prealloc_size = 590558003, real_size = 330805800 2024/07/08-15:16:04.271890 139818746500672 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627124.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:16:06.452275 139818746500672 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627124.sst): second pass time = 5.15's, 95.986'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.18's, 158.738'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 13212.617'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.37's, 934.499'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 90944.329'MB/sec zip my value time = 5.15's, unzip length = 0.494'GB zip my value throughput = 95.986'MB/sec zip pipeline throughput = 95.986'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 3.076271 GB avg = 0.191 KB (by entry num) total key len = 0.257698 GB avg = 0.016 KB total ukey len = 0.128849 GB avg = 0.008 KB total ukey num = 0.016106130 Billion total entry num = 0.016106130 Billion write speed all = 0.173803826 MB/sec (with seq num) write speed all = 0.167086770 MB/sec (without seq num) 2024/07/08-15:16:06.828892 139818746500672 [compaction_job.cc:1944] [default] [JOB 2442] Generated table #12627124: 2684355 keys, 346122699 bytes, temperature: kWarm 2024/07/08-15:16:06.828957 139818746500672 EVENT_LOG_v1 {"time_micros": 1720422966828926, "cf_name": "default", "job": 2442, "event": "table_file_creation", "file_number": 12627124, "file_size": 346122699, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142200830959, "largest_seqno": 147180687708, "table_properties": {"data_size": 329714880, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422950, "oldest_key_time": 0, "file_creation_time": 1720422950, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12627124, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:16:06.932498 139818746500672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627125.sst) /dev/shm/ToplingTemp/Topling-14573-vJp826 2024/07/08-15:16:12.878396 139818746500672 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627125.sst): first pass time = 5.95's, 90.293'MB/sec 2024/07/08-15:16:17.963055 139818746500672 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627125.sst): old prealloc_size = 590558003, real_size = 330809224 2024/07/08-15:16:17.963139 139818746500672 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627125.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:16:20.152920 139818746500672 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627125.sst): second pass time = 5.08's, 97.181'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.19's, 158.058'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 13924.491'MB/sec (index lex order gen) rebuild zvType time = 1.82's, 0.370'MB/sec write SST data time = 0.37's, 925.602'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 96386.968'MB/sec zip my value time = 5.08's, unzip length = 0.494'GB zip my value throughput = 97.181'MB/sec zip pipeline throughput = 97.181'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6676 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 3.588983 GB avg = 0.191 KB (by entry num) total key len = 0.300648 GB avg = 0.016 KB total ukey len = 0.150324 GB avg = 0.008 KB total ukey num = 0.018790485 Billion total entry num = 0.018790485 Billion write speed all = 0.202626408 MB/sec (with seq num) write speed all = 0.194795436 MB/sec (without seq num) 2024/07/08-15:16:20.533582 139818746500672 [compaction_job.cc:1944] [default] [JOB 2442] Generated table #12627125: 2684355 keys, 346126155 bytes, temperature: kWarm 2024/07/08-15:16:20.533651 139818746500672 EVENT_LOG_v1 {"time_micros": 1720422980533617, "cf_name": "default", "job": 2442, "event": "table_file_creation", "file_number": 12627125, "file_size": 346126155, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142324195448, "largest_seqno": 147180687707, "table_properties": {"data_size": 329718304, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422966, "oldest_key_time": 0, "file_creation_time": 1720422966, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12627125, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:16:20.640526 139818746500672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627126.sst) /dev/shm/ToplingTemp/Topling-14573-h5zg67 2024/07/08-15:16:25.932009 139818746500672 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627126.sst): first pass time = 5.29's, 45.715'MB/sec 2024/07/08-15:16:28.238167 139818746500672 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627126.sst): old prealloc_size = 590558003, real_size = 157531728 2024/07/08-15:16:28.238259 139818746500672 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627126.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1209484 prefix = 4 raw-key = 0.0048 GB zip-key = 0.0002 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.2225 GB zip-val = 0.1570 GB avg-val = 184.00 avg-zval = 129.84 2024/07/08-15:16:29.196417 139818746500672 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627126.sst): second pass time = 2.30's, 96.559'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 171.596'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 5640.943'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.386'MB/sec write SST data time = 0.18's, 937.444'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.211'MB, throughput = 52738.211'MB/sec zip my value time = 2.30's, unzip length = 0.223'GB zip my value throughput = 96.559'MB/sec zip pipeline throughput = 96.559'MB/sec entries = 1209484 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 129.84 usrkeys = 1209484 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 129.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1209484 bytes = 4989136 } seq expand size = 8466388 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0097 GB value = 0.2225 GB dict = 2.21 MB all = 0.2419 GB } __ZipSize{ index = 0.0002 GB value = 0.1570 GB dict = 2.21 MB all = 0.1644 GB } UnZip/Zip{ index = 51.1280 value = 1.4171 dict = 1.00 all = 1.4711 } Zip/UnZip{ index = 0.0196 value = 0.7057 dict = 1.00 all = 0.6798 } ---------------------------- total value len = 3.819994 GB avg = 0.191 KB (by entry num) total key len = 0.320000 GB avg = 0.016 KB total ukey len = 0.160000 GB avg = 0.008 KB total ukey num = 0.019999969 Billion total entry num = 0.019999969 Billion write speed all = 0.215567272 MB/sec (with seq num) write speed all = 0.207236169 MB/sec (without seq num) 2024/07/08-15:16:29.380018 139818746500672 [compaction_job.cc:1944] [default] [JOB 2442] Generated table #12627126: 1209484 keys, 164430485 bytes, temperature: kWarm 2024/07/08-15:16:29.380090 139818746500672 EVENT_LOG_v1 {"time_micros": 1720422989380053, "cf_name": "default", "job": 2442, "event": "table_file_creation", "file_number": 12627126, "file_size": 164430485, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142443275204, "largest_seqno": 147180687297, "table_properties": {"data_size": 157039952, "index_size": 189248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19351744, "raw_average_key_size": 16, "raw_value_size": 222545056, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1209484, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422980, "oldest_key_time": 0, "file_creation_time": 1720422980, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12627126, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:16:29.435850 139818746500672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627127.sst) /dev/shm/ToplingTemp/Topling-14573-gJA2ca 2024/07/08-15:16:36.177696 139818746500672 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627127.sst): first pass time = 6.74's, 79.633'MB/sec 2024/07/08-15:16:53.638011 139818746500672 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627127.sst): old prealloc_size = 590558003, real_size = 330815368 2024/07/08-15:16:53.638087 139818746500672 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627127.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:16:56.064088 139818746500672 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627127.sst): second pass time = 17.46's, 28.292'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.43's, 142.672'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 14082.405'MB/sec (index lex order gen) rebuild zvType time = 1.97's, 0.341'MB/sec write SST data time = 0.46's, 756.568'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 78681.633'MB/sec zip my value time = 17.46's, unzip length = 0.494'GB zip my value throughput = 28.292'MB/sec zip pipeline throughput = 145.041'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6676 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 4.332706 GB avg = 0.191 KB (by entry num) total key len = 0.362949 GB avg = 0.016 KB total ukey len = 0.181475 GB avg = 0.008 KB total ukey num = 0.022684324 Billion total entry num = 0.022684324 Billion write speed all = 0.244158696 MB/sec (with seq num) write speed all = 0.234722611 MB/sec (without seq num) 2024/07/08-15:16:56.450268 139818746500672 [compaction_job.cc:1944] [default] [JOB 2442] Generated table #12627127: 2684355 keys, 346132299 bytes, temperature: kWarm 2024/07/08-15:16:56.450347 139818746500672 EVENT_LOG_v1 {"time_micros": 1720423016450303, "cf_name": "default", "job": 2442, "event": "table_file_creation", "file_number": 12627127, "file_size": 346132299, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142183816867, "largest_seqno": 147180687056, "table_properties": {"data_size": 329724448, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720422989, "oldest_key_time": 0, "file_creation_time": 1720422989, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12627127, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:16:56.558395 139818746500672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627128.sst) /dev/shm/ToplingTemp/Topling-14573-c2E5F6 2024/07/08-15:16:57.518493 139818746500672 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627128.sst): first pass time = 0.96's, 176.151'MB/sec 2024/07/08-15:16:59.025717 139818746500672 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627128.sst): old prealloc_size = 590558003, real_size = 113074672 2024/07/08-15:16:59.025783 139818746500672 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627128.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 845566 prefix = 4 raw-key = 0.0034 GB zip-key = 0.0001 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.1556 GB zip-val = 0.1127 GB avg-val = 184.00 avg-zval = 133.32 2024/07/08-15:16:59.761244 139818746500672 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00/12627128.sst): second pass time = 1.51's, 103.303'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 160.311'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 4148.482'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.346'MB/sec write SST data time = 0.12's, 953.508'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.558'MB, throughput = 27956.430'MB/sec zip my value time = 1.51's, unzip length = 0.156'GB zip my value throughput = 103.303'MB/sec zip pipeline throughput = 103.303'MB/sec entries = 845566 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 133.32 usrkeys = 845566 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 133.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 845566 bytes = 3487968 } seq expand size = 5918962 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0068 GB value = 0.1556 GB dict = 1.56 MB all = 0.1691 GB } __ZipSize{ index = 0.0001 GB value = 0.1127 GB dict = 1.56 MB all = 0.1179 GB } UnZip/Zip{ index = 51.0361 value = 1.3801 dict = 1.00 all = 1.4343 } Zip/UnZip{ index = 0.0196 value = 0.7246 dict = 1.00 all = 0.6972 } ---------------------------- total value len = 4.494209 GB avg = 0.191 KB (by entry num) total key len = 0.376478 GB avg = 0.016 KB total ukey len = 0.188239 GB avg = 0.008 KB total ukey num = 0.023529890 Billion total entry num = 0.023529890 Billion write speed all = 0.253211119 MB/sec (with seq num) write speed all = 0.243425182 MB/sec (without seq num) 2024/07/08-15:16:59.897833 139818746500672 [compaction_job.cc:1944] [default] [JOB 2442] Generated table #12627128: 845566 keys, 117910204 bytes, temperature: kWarm 2024/07/08-15:16:59.897903 139818746500672 EVENT_LOG_v1 {"time_micros": 1720423019897867, "cf_name": "default", "job": 2442, "event": "table_file_creation", "file_number": 12627128, "file_size": 117910204, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142135996218, "largest_seqno": 147180686619, "table_properties": {"data_size": 112730560, "index_size": 132544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13529056, "raw_average_key_size": 16, "raw_value_size": 155584144, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 845566, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720423016, "oldest_key_time": 0, "file_creation_time": 1720423016, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12627128, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:16:59.933588 139818746500672 [compaction_job.cc:699] job-02442: subcompact[0], size: 3.051292 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02442/att-00 [12627119,12627120,12627121,12627122,12627123,12627124,12627125,12627126,12627127,12627128] 2024/07/08-15:17:00.261327 139818746500672 (Original Log Time 2024/07/08-15:14:33.160343) EVENT_LOG_v1 {"time_micros": 1720422873160288, "job": 2442, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L4": [12625977, 12625978, 12625979, 12625980, 12625981, 12625982, 12625983, 12625984, 12625985, 12625986], "files_L5": [12627115, 12627116, 12608620, 12625558, 12435388, 12583208, 12618546, 12509554, 12606818, 12606199, 12617089, 12574465, 12625559, 12603650, 12621854, 12609441], "score": -1, "input_data_size": 4851229580, "oldest_snapshot_seqno": 147212954258} 2024/07/08-15:17:00.261334 139818746500672 (Original Log Time 2024/07/08-15:17:00.254066) [compaction_job.cc:2018] [default] [JOB 2442] Compacted 10@4 + 16@5 files to L5 => 3051291817 bytes 2024/07/08-15:17:00.261336 139818746500672 (Original Log Time 2024/07/08-15:17:00.261231) [compaction_job.cc:1220] [default] compacted to: files[0 0 0 0 0 10 0] max score 0.00, MB/sec: 33.1 rd, 20.8 wr, level 5, files in(10, 16) out(10 +0 blob) MB in(1673.9, 2952.6 +0.0 blob) out(2909.9 +0.0 blob), read-write-amplify(4.5) write-amplify(1.7) OK, records in: 36246661, records dropped: 12716771 output_compression: Snappy 2024/07/08-15:17:00.261339 139818746500672 (Original Log Time 2024/07/08-15:17:00.261282) EVENT_LOG_v1 {"time_micros": 1720423020261259, "cf": "default", "job": 2442, "event": "compaction_finished", "compaction_time_micros": 146773256, "compaction_time_cpu_micros": 52106598, "output_level": 5, "num_output_files": 10, "total_output_size": 3051291817, "num_input_records": 36246661, "num_output_records": 23529890, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 1784856782, "file_range_sync_nanos": 0, "file_fsync_nanos": 7954, "file_prepare_write_nanos": 38177, "lsm_state": [0, 0, 0, 0, 0, 10, 0]} 2024/07/08-15:17:00.392982 139818746500672 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"a007a635-5c0a-4b4a-be96-b9c695cb016b","dbStarts":1720420896,"starts":1720422873,"executesMs":147224,"compactionJobId":2442,"attempt":0,"compactionInputRawBytes":7249332200,"compactionInputZipBytes":4851229580,"compactionOutputRawBytes":0,"compactionOutputZipBytes":3051291817,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-7d5462cf62d8f62ef1b4500bc8f7d09f-524ed28c17cae2c1-00"}}, response = 2024/07/08-15:17:00.393093 139818746500672 INFO dcompact_worker.cpp:1456: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-02442/att-00: olev 5, work 147.225 s, result 6.977 ms, install 7.311 ms, input{raw 6.751 GiB zip 4.518 GiB}