2024/08/20-00:33:21.326361 140500271638080 DEBG dcompact_worker.cpp:2211: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00: fork to child time = 0.014272 sec 2024/08/20-00:33:21.327613 140500271638080 DEBG dcompact_worker.cpp:1011: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00 2024/08/20-00:33:21.339305 140500271638080 DEBG dcompact_worker.cpp:1031: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00 2024/08/20-00:33:21.341044 140500271638080 TRAC dcompact_worker.cpp:1081: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/08/20-00:33:21.341445 140500271638080 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/08/20-00:33:21.393590 140500271638080 [version_set.cc:6139] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-19T23.41.30/.rocksdb/job-02410/att-00/MANIFEST-14619535 2024/08/20-00:33:21.443835 140500271638080 [version_set.cc:6182] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-19T23.41.30/.rocksdb/job-02410/att-00/MANIFEST-14619535 succeeded,manifest_file_number is 14619535, next_file_number is 14622273, last_sequence is 189641432262, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 14622171 2024/08/20-00:33:21.443851 140500271638080 [version_set.cc:6197] Column family [default] (ID 0), log number is 0 2024/08/20-00:33:21.444080 140500271638080 [version_set.cc:5652] Creating manifest 14622273 2024/08/20-00:33:21.469967 140500271638080 DEBG dcompact_worker.cpp:1215: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00: bottommost_level: fake = 1, rpc = 0 2024/08/20-00:33:21.487254 140500271638080 [compaction_job.cc:2433] [default] [JOB 2410] Compacting 17@3 + 21@4 files to L4, score -1.00, subcompactions 1 : 1 2024/08/20-00:33:21.487304 140500271638080 [compaction_job.cc:2441] [default]: Compaction start summary: Base version 1218 Base level 3, inputs: [14622118(90MB) 14622119(90MB) 14622121(90MB) 14622122(74MB) 14622211(91MB) 14622212(90MB) 14622214(90MB) 14622215(90MB) 14622216(22MB) 14622128(34MB) 14622129(90MB) 14622130(91MB) 14622131(90MB) 14622132(90MB) 14622133(31MB) 14622134(91MB) 14622135(90MB)], [14622271(52MB) 14620643(152MB) 14622254(159MB) 14621777(146MB) 14620022(129MB) 14621663(172MB) 14621664(57MB) 14620023(149MB) 14622257(160MB) 14620428(172MB) 14620429(70MB) 14619778(159MB) 14621662(152MB) 14621033(160MB) 14620532(148MB) 14620943(152MB) 14621132(172MB) 14621133(58MB) 14620641(172MB) 14620642(16MB) 14619796(172MB)] 2024/08/20-00:33:21.512102 140500271638080 [top_zip_table.cc:59] core git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-00:33:21.512164 140500271638080 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-00:33:21.512199 140500271638080 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/08/20-00:33:21.512215 140500271638080 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 7f80399bfbd2071b6e0f5683e8ab7225b8e6e394 Author: leipeng Date: 2024-01-19 14:52:30 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/08/20-00:33:21.512239 140500271638080 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit 1240987ca13d0fe562c9b9c037155079f0877840 Author: leipeng Date: 2024-01-19 15:07:04 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/08/20-00:33:22.546277 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622274.sst) /dev/shm/ToplingTemp/Topling-78558-xMoXKm 2024/08/20-00:33:26.520054 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622274.sst): first pass time = 3.97's, 67.553'MB/sec 2024/08/20-00:33:30.281767 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622274.sst): old prealloc_size = 295279001, real_size = 173785920 2024/08/20-00:33:30.281888 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622274.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1729 GB avg-val = 184.00 avg-zval = 128.85 2024/08/20-00:33:32.012383 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622274.sst): second pass time = 3.76's, 65.692'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.73's, 104.352'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 9244.821'MB/sec (index lex order gen) rebuild zvType time = 1.46's, 0.230'MB/sec write SST data time = 0.27's, 658.010'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 62705.282'MB/sec zip my value time = 3.76's, unzip length = 0.247'GB zip my value throughput = 65.692'MB/sec zip pipeline throughput = 66.294'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.85 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1729 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.2343 value = 1.4280 dict = 1.00 all = 1.4864 } Zip/UnZip{ index = 0.0471 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 62.553106 GB avg = 0.180 KB (by entry num) total key len = 6.810233 GB avg = 0.020 KB total ukey len = 3.200245 GB avg = 0.008 KB total ukey num = 0.390675739 Billion total entry num = 0.347553747 Billion write speed all = 2.755705461 MB/sec (with seq num) write speed all = 2.645242985 MB/sec (without seq num) 2024/08/20-00:33:32.218630 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622274: 1342178 keys, 180593453 bytes, temperature: kWarm 2024/08/20-00:33:32.218736 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085212218675, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622274, "file_size": 180593453, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410453968, "largest_seqno": 189634998158, "table_properties": {"data_size": 172944528, "index_size": 505664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085202, "oldest_key_time": 0, "file_creation_time": 1724085202, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622274, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:33:32.313436 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622275.sst) /dev/shm/ToplingTemp/Topling-78558-6SRiqn 2024/08/20-00:33:35.888517 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622275.sst): first pass time = 3.58's, 75.086'MB/sec 2024/08/20-00:33:41.522717 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622275.sst): old prealloc_size = 295279001, real_size = 173785360 2024/08/20-00:33:41.522870 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622275.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1729 GB avg-val = 184.00 avg-zval = 128.85 2024/08/20-00:33:42.639538 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622275.sst): second pass time = 5.63's, 43.846'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 161.703'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15446.860'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.370'MB/sec write SST data time = 0.21's, 858.896'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 41056.114'MB/sec zip my value time = 5.63's, unzip length = 0.247'GB zip my value throughput = 43.846'MB/sec zip pipeline throughput = 104.795'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.85 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1729 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.2289 value = 1.4280 dict = 1.00 all = 1.4864 } Zip/UnZip{ index = 0.0471 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 62.809462 GB avg = 0.180 KB (by entry num) total key len = 6.831707 GB avg = 0.020 KB total ukey len = 3.210982 GB avg = 0.008 KB total ukey num = 0.392017917 Billion total entry num = 0.348895925 Billion write speed all = 2.765575648 MB/sec (with seq num) write speed all = 2.654733387 MB/sec (without seq num) 2024/08/20-00:33:42.841220 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622275: 1342178 keys, 180592877 bytes, temperature: kWarm 2024/08/20-00:33:42.841277 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085222841247, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622275, "file_size": 180592877, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410454339, "largest_seqno": 189634998161, "table_properties": {"data_size": 172943808, "index_size": 505792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085212, "oldest_key_time": 0, "file_creation_time": 1724085212, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622275, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:33:42.915369 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622276.sst) /dev/shm/ToplingTemp/Topling-78558-7Gtzon 2024/08/20-00:33:47.536996 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622276.sst): first pass time = 4.62's, 58.083'MB/sec 2024/08/20-00:33:53.026594 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622276.sst): old prealloc_size = 295279001, real_size = 173794608 2024/08/20-00:33:53.026677 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622276.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:33:54.213008 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622276.sst): second pass time = 5.49's, 45.002'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.19's, 152.226'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15998.987'MB/sec (index lex order gen) rebuild zvType time = 0.99's, 0.339'MB/sec write SST data time = 0.20's, 923.190'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 53052.348'MB/sec zip my value time = 5.49's, unzip length = 0.247'GB zip my value throughput = 45.002'MB/sec zip pipeline throughput = 108.482'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.2370 value = 1.4279 dict = 1.00 all = 1.4863 } Zip/UnZip{ index = 0.0471 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 63.065818 GB avg = 0.180 KB (by entry num) total key len = 6.853182 GB avg = 0.020 KB total ukey len = 3.221720 GB avg = 0.008 KB total ukey num = 0.393360095 Billion total entry num = 0.350238103 Billion write speed all = 2.775333257 MB/sec (with seq num) write speed all = 2.664115710 MB/sec (without seq num) 2024/08/20-00:33:54.411476 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622276: 1342178 keys, 180602157 bytes, temperature: kWarm 2024/08/20-00:33:54.411547 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085234411514, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622276, "file_size": 180602157, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410453970, "largest_seqno": 189634998162, "table_properties": {"data_size": 172953296, "index_size": 505600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085222, "oldest_key_time": 0, "file_creation_time": 1724085222, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622276, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:33:54.478354 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622277.sst) /dev/shm/ToplingTemp/Topling-78558-NH4A0o 2024/08/20-00:33:56.034421 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622277.sst): first pass time = 1.56's, 172.514'MB/sec 2024/08/20-00:33:59.583499 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622277.sst): old prealloc_size = 295279001, real_size = 173790768 2024/08/20-00:33:59.583587 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622277.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1729 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:34:00.658015 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622277.sst): second pass time = 3.55's, 69.622'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 168.078'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 16089.533'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.387'MB/sec write SST data time = 0.21's, 869.970'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 54509.856'MB/sec zip my value time = 3.55's, unzip length = 0.247'GB zip my value throughput = 69.622'MB/sec zip pipeline throughput = 113.178'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1729 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.2182 value = 1.4279 dict = 1.00 all = 1.4864 } Zip/UnZip{ index = 0.0471 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 63.322174 GB avg = 0.180 KB (by entry num) total key len = 6.874657 GB avg = 0.020 KB total ukey len = 3.232457 GB avg = 0.008 KB total ukey num = 0.394702273 Billion total entry num = 0.351580281 Billion write speed all = 2.785648712 MB/sec (with seq num) write speed all = 2.674033512 MB/sec (without seq num) 2024/08/20-00:34:00.871130 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622277: 1342178 keys, 180598317 bytes, temperature: kWarm 2024/08/20-00:34:00.871217 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085240871172, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622277, "file_size": 180598317, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410453976, "largest_seqno": 189634997954, "table_properties": {"data_size": 172948976, "index_size": 506048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085234, "oldest_key_time": 0, "file_creation_time": 1724085234, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622277, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:00.963285 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622278.sst) /dev/shm/ToplingTemp/Topling-78558-1xDffp 2024/08/20-00:34:03.812758 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622278.sst): first pass time = 2.85's, 94.207'MB/sec 2024/08/20-00:34:06.744161 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622278.sst): old prealloc_size = 295279001, real_size = 173790592 2024/08/20-00:34:06.744236 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622278.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1729 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:34:07.797692 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622278.sst): second pass time = 2.93's, 84.300'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 171.424'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 16062.578'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.389'MB/sec write SST data time = 0.19's, 947.852'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 64071.084'MB/sec zip my value time = 2.93's, unzip length = 0.247'GB zip my value throughput = 84.300'MB/sec zip pipeline throughput = 84.300'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1729 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.2558 value = 1.4279 dict = 1.00 all = 1.4864 } Zip/UnZip{ index = 0.0470 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 63.578530 GB avg = 0.180 KB (by entry num) total key len = 6.896132 GB avg = 0.020 KB total ukey len = 3.243195 GB avg = 0.008 KB total ukey num = 0.396044451 Billion total entry num = 0.352922459 Billion write speed all = 2.795881834 MB/sec (with seq num) write speed all = 2.683872273 MB/sec (without seq num) 2024/08/20-00:34:07.998646 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622278: 1342178 keys, 180598125 bytes, temperature: kWarm 2024/08/20-00:34:07.998718 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085247998684, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622278, "file_size": 180598125, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410453969, "largest_seqno": 189634998160, "table_properties": {"data_size": 172949680, "index_size": 505152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085240, "oldest_key_time": 0, "file_creation_time": 1724085240, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622278, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:08.059405 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622279.sst) /dev/shm/ToplingTemp/Topling-78558-igvW0m 2024/08/20-00:34:10.118249 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622279.sst): first pass time = 2.06's, 24.007'MB/sec 2024/08/20-00:34:10.478724 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622279.sst): old prealloc_size = 295279001, real_size = 36085144 2024/08/20-00:34:10.478780 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622279.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 247128 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0001 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.0455 GB zip-val = 0.0359 GB avg-val = 184.00 avg-zval = 145.39 2024/08/20-00:34:10.678548 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622279.sst): second pass time = 0.36's, 126.415'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 186.947'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3072.321'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.379'MB/sec write SST data time = 0.04's, 1019.905'MB/sec dict compress time = 0.00's, 465704.000'MB/sec z-dict build time = 0.00's, sample length = 0.466'MB, throughput = 13781.895'MB/sec zip my value time = 0.36's, unzip length = 0.045'GB zip my value throughput = 126.415'MB/sec zip pipeline throughput = 126.415'MB/sec entries = 247128 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 145.39 usrkeys = 247128 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 145.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 247128 bytes = 864960 } seq expand size = 1729896 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0020 GB value = 0.0455 GB dict = 0.47 MB all = 0.0494 GB } __ZipSize{ index = 0.0001 GB value = 0.0359 GB dict = 0.47 MB all = 0.0374 GB } UnZip/Zip{ index = 21.2018 value = 1.2656 dict = 1.00 all = 1.3231 } Zip/UnZip{ index = 0.0472 value = 0.7902 dict = 1.00 all = 0.7558 } ---------------------------- total value len = 63.625731 GB avg = 0.180 KB (by entry num) total key len = 6.900086 GB avg = 0.020 KB total ukey len = 3.245172 GB avg = 0.008 KB total ukey num = 0.396291579 Billion total entry num = 0.353169587 Billion write speed all = 2.797591546 MB/sec (with seq num) write speed all = 2.685516361 MB/sec (without seq num) 2024/08/20-00:34:10.723368 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622279: 247128 keys, 37355131 bytes, temperature: kWarm 2024/08/20-00:34:10.723435 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085250723402, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622279, "file_size": 37355131, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410458221, "largest_seqno": 189634997648, "table_properties": {"data_size": 35929984, "index_size": 93248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3954048, "raw_average_key_size": 16, "raw_value_size": 45471552, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 247128, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085248, "oldest_key_time": 0, "file_creation_time": 1724085248, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622279, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:10.759502 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622280.sst) /dev/shm/ToplingTemp/Topling-78558-1JQWio 2024/08/20-00:34:11.968336 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622280.sst): first pass time = 1.21's, 222.070'MB/sec 2024/08/20-00:34:14.702944 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622280.sst): old prealloc_size = 295279001, real_size = 173787648 2024/08/20-00:34:14.703006 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622280.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.37 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:34:15.765240 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622280.sst): second pass time = 2.73's, 90.353'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 170.004'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15906.134'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.387'MB/sec write SST data time = 0.20's, 923.574'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 59201.744'MB/sec zip my value time = 2.73's, unzip length = 0.247'GB zip my value throughput = 90.353'MB/sec zip pipeline throughput = 90.353'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.37 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.37 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.5231 value = 1.4279 dict = 1.00 all = 1.4864 } Zip/UnZip{ index = 0.0465 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 63.882087 GB avg = 0.180 KB (by entry num) total key len = 6.921561 GB avg = 0.020 KB total ukey len = 3.255909 GB avg = 0.008 KB total ukey num = 0.397633757 Billion total entry num = 0.354511765 Billion write speed all = 2.808045837 MB/sec (with seq num) write speed all = 2.695567419 MB/sec (without seq num) 2024/08/20-00:34:15.965704 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622280: 1342178 keys, 180595245 bytes, temperature: kWarm 2024/08/20-00:34:15.965776 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085255965742, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622280, "file_size": 180595245, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410455974, "largest_seqno": 189640132532, "table_properties": {"data_size": 172953056, "index_size": 498880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085250, "oldest_key_time": 0, "file_creation_time": 1724085250, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622280, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:16.030508 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622281.sst) /dev/shm/ToplingTemp/Topling-78558-g0soNp 2024/08/20-00:34:18.770111 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622281.sst): first pass time = 2.74's, 97.985'MB/sec 2024/08/20-00:34:21.430627 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622281.sst): old prealloc_size = 295279001, real_size = 173787024 2024/08/20-00:34:21.430706 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622281.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.37 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:34:22.487705 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622281.sst): second pass time = 2.66's, 92.891'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 170.846'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 16557.763'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.388'MB/sec write SST data time = 0.19's, 943.968'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 50337.514'MB/sec zip my value time = 2.66's, unzip length = 0.247'GB zip my value throughput = 92.891'MB/sec zip pipeline throughput = 92.891'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.37 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.37 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.5093 value = 1.4279 dict = 1.00 all = 1.4864 } Zip/UnZip{ index = 0.0465 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 64.138443 GB avg = 0.180 KB (by entry num) total key len = 6.943036 GB avg = 0.020 KB total ukey len = 3.266646 GB avg = 0.008 KB total ukey num = 0.398975935 Billion total entry num = 0.355853943 Billion write speed all = 2.818313111 MB/sec (with seq num) write speed all = 2.705438945 MB/sec (without seq num) 2024/08/20-00:34:22.688272 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622281: 1342178 keys, 180594605 bytes, temperature: kWarm 2024/08/20-00:34:22.688343 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085262688307, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622281, "file_size": 180594605, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410454435, "largest_seqno": 189640132533, "table_properties": {"data_size": 172952112, "index_size": 499200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085256, "oldest_key_time": 0, "file_creation_time": 1724085256, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622281, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:22.750900 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622282.sst) /dev/shm/ToplingTemp/Topling-78558-se75to 2024/08/20-00:34:25.756012 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622282.sst): first pass time = 3.01's, 89.328'MB/sec 2024/08/20-00:34:28.461164 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622282.sst): old prealloc_size = 295279001, real_size = 173792504 2024/08/20-00:34:28.461246 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622282.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.37 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:34:29.524266 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622282.sst): second pass time = 2.70's, 91.348'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 169.882'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15435.923'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.387'MB/sec write SST data time = 0.20's, 922.172'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 61790.698'MB/sec zip my value time = 2.70's, unzip length = 0.247'GB zip my value throughput = 91.348'MB/sec zip pipeline throughput = 91.348'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.37 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.37 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.5120 value = 1.4279 dict = 1.00 all = 1.4864 } Zip/UnZip{ index = 0.0465 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 64.394799 GB avg = 0.180 KB (by entry num) total key len = 6.964511 GB avg = 0.019 KB total ukey len = 3.277384 GB avg = 0.008 KB total ukey num = 0.400318113 Billion total entry num = 0.357196121 Billion write speed all = 2.828539698 MB/sec (with seq num) write speed all = 2.715271404 MB/sec (without seq num) 2024/08/20-00:34:29.723596 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622282: 1342178 keys, 180600045 bytes, temperature: kWarm 2024/08/20-00:34:29.723664 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085269723631, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622282, "file_size": 180600045, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410455045, "largest_seqno": 189640132531, "table_properties": {"data_size": 172957632, "index_size": 499136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085262, "oldest_key_time": 0, "file_creation_time": 1724085262, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622282, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:29.805211 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622283.sst) /dev/shm/ToplingTemp/Topling-78558-Q1jePp 2024/08/20-00:34:31.657915 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622283.sst): first pass time = 1.85's, 144.893'MB/sec 2024/08/20-00:34:34.293644 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622283.sst): old prealloc_size = 295279001, real_size = 173794712 2024/08/20-00:34:34.293705 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622283.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.37 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:34:35.354562 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622283.sst): second pass time = 2.63's, 93.763'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 170.235'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 16340.247'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.388'MB/sec write SST data time = 0.20's, 923.360'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 50271.260'MB/sec zip my value time = 2.63's, unzip length = 0.247'GB zip my value throughput = 93.763'MB/sec zip pipeline throughput = 93.763'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.37 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.37 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.4982 value = 1.4279 dict = 1.00 all = 1.4863 } Zip/UnZip{ index = 0.0465 value = 0.7004 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 64.651155 GB avg = 0.180 KB (by entry num) total key len = 6.985985 GB avg = 0.019 KB total ukey len = 3.288121 GB avg = 0.008 KB total ukey num = 0.401660291 Billion total entry num = 0.358538299 Billion write speed all = 2.838896283 MB/sec (with seq num) write speed all = 2.725228648 MB/sec (without seq num) 2024/08/20-00:34:35.559287 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622283: 1342178 keys, 180602285 bytes, temperature: kWarm 2024/08/20-00:34:35.559359 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085275559322, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622283, "file_size": 180602285, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410454432, "largest_seqno": 189640132530, "table_properties": {"data_size": 172959520, "index_size": 499456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085269, "oldest_key_time": 0, "file_creation_time": 1724085269, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622283, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:35.622233 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622284.sst) /dev/shm/ToplingTemp/Topling-78558-21Uznq 2024/08/20-00:34:38.819969 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622284.sst): first pass time = 3.20's, 83.947'MB/sec 2024/08/20-00:34:41.488674 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622284.sst): old prealloc_size = 295279001, real_size = 173798040 2024/08/20-00:34:41.488757 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622284.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.37 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.87 2024/08/20-00:34:42.545303 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622284.sst): second pass time = 2.67's, 92.591'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 170.928'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15715.023'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.388'MB/sec write SST data time = 0.19's, 944.724'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 58177.404'MB/sec zip my value time = 2.67's, unzip length = 0.247'GB zip my value throughput = 92.591'MB/sec zip pipeline throughput = 92.591'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.37 avg-val = 184.00 avg-zval = 128.87 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.37 avg-val = 191.00 avg-zval = 128.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.5010 value = 1.4278 dict = 1.00 all = 1.4863 } Zip/UnZip{ index = 0.0465 value = 0.7004 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 64.907511 GB avg = 0.180 KB (by entry num) total key len = 7.007460 GB avg = 0.019 KB total ukey len = 3.298859 GB avg = 0.008 KB total ukey num = 0.403002469 Billion total entry num = 0.359880477 Billion write speed all = 2.849094514 MB/sec (with seq num) write speed all = 2.735033870 MB/sec (without seq num) 2024/08/20-00:34:42.746456 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622284: 1342178 keys, 180605613 bytes, temperature: kWarm 2024/08/20-00:34:42.746528 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085282746491, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622284, "file_size": 180605613, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410455236, "largest_seqno": 189640132115, "table_properties": {"data_size": 172962928, "index_size": 499392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085275, "oldest_key_time": 0, "file_creation_time": 1724085275, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622284, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:42.812048 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622285.sst) /dev/shm/ToplingTemp/Topling-78558-XRoi7l 2024/08/20-00:34:44.942606 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622285.sst): first pass time = 2.13's, 125.996'MB/sec 2024/08/20-00:34:47.596039 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622285.sst): old prealloc_size = 295279001, real_size = 173803520 2024/08/20-00:34:47.596104 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622285.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.87 2024/08/20-00:34:48.661429 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622285.sst): second pass time = 2.65's, 93.131'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 169.528'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15439.034'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.386'MB/sec write SST data time = 0.20's, 921.877'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 49706.977'MB/sec zip my value time = 2.65's, unzip length = 0.247'GB zip my value throughput = 93.131'MB/sec zip pipeline throughput = 93.131'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.87 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.2585 value = 1.4278 dict = 1.00 all = 1.4863 } Zip/UnZip{ index = 0.0470 value = 0.7004 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 65.163867 GB avg = 0.180 KB (by entry num) total key len = 7.028935 GB avg = 0.019 KB total ukey len = 3.309596 GB avg = 0.008 KB total ukey num = 0.404344647 Billion total entry num = 0.361222655 Billion write speed all = 2.859408638 MB/sec (with seq num) write speed all = 2.744950338 MB/sec (without seq num) 2024/08/20-00:34:48.861003 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622285: 1342178 keys, 180611053 bytes, temperature: kWarm 2024/08/20-00:34:48.861068 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085288861036, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622285, "file_size": 180611053, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410455239, "largest_seqno": 189640129516, "table_properties": {"data_size": 172962688, "index_size": 505088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085282, "oldest_key_time": 0, "file_creation_time": 1724085282, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622285, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:48.925128 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622286.sst) /dev/shm/ToplingTemp/Topling-78558-N9OPTp 2024/08/20-00:34:50.996872 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622286.sst): first pass time = 2.07's, 32.208'MB/sec 2024/08/20-00:34:51.475737 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622286.sst): old prealloc_size = 295279001, real_size = 47617408 2024/08/20-00:34:51.475801 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622286.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 333630 prefix = 4 raw-key = 0.0013 GB zip-key = 0.0001 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.0614 GB zip-val = 0.0474 GB avg-val = 184.00 avg-zval = 142.10 2024/08/20-00:34:51.743335 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622286.sst): second pass time = 0.48's, 128.414'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.27's, 184.359'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 4019.808'MB/sec (index lex order gen) rebuild zvType time = 0.22's, 0.382'MB/sec write SST data time = 0.05's, 1004.506'MB/sec dict compress time = 0.00's, 630200.000'MB/sec z-dict build time = 0.00's, sample length = 0.630'MB, throughput = 15321.032'MB/sec zip my value time = 0.48's, unzip length = 0.061'GB zip my value throughput = 128.414'MB/sec zip pipeline throughput = 128.414'MB/sec entries = 333630 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 142.10 usrkeys = 333630 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 142.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 333630 bytes = 1167712 } seq expand size = 2335410 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0027 GB value = 0.0614 GB dict = 0.63 MB all = 0.0667 GB } __ZipSize{ index = 0.0001 GB value = 0.0474 GB dict = 0.63 MB all = 0.0493 GB } UnZip/Zip{ index = 21.2125 value = 1.2949 dict = 1.00 all = 1.3526 } Zip/UnZip{ index = 0.0471 value = 0.7723 dict = 1.00 all = 0.7393 } ---------------------------- total value len = 65.227591 GB avg = 0.180 KB (by entry num) total key len = 7.034273 GB avg = 0.019 KB total ukey len = 3.312265 GB avg = 0.008 KB total ukey num = 0.404678277 Billion total entry num = 0.361556285 Billion write speed all = 2.861794685 MB/sec (with seq num) write speed all = 2.747244652 MB/sec (without seq num) 2024/08/20-00:34:51.802735 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622286: 333630 keys, 49332939 bytes, temperature: kWarm 2024/08/20-00:34:51.802805 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085291802772, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622286, "file_size": 49332939, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410454441, "largest_seqno": 189634996029, "table_properties": {"data_size": 47408016, "index_size": 125824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5338080, "raw_average_key_size": 16, "raw_value_size": 61387920, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 333630, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085288, "oldest_key_time": 0, "file_creation_time": 1724085288, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622286, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:51.830296 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622287.sst) /dev/shm/ToplingTemp/Topling-78558-nYdHGm 2024/08/20-00:34:53.105276 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622287.sst): first pass time = 1.27's, 210.550'MB/sec 2024/08/20-00:34:55.823626 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622287.sst): old prealloc_size = 295279001, real_size = 173795616 2024/08/20-00:34:55.823691 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622287.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1729 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:34:56.888472 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622287.sst): second pass time = 2.72's, 90.913'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 169.606'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15949.162'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.385'MB/sec write SST data time = 0.19's, 932.024'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 54496.489'MB/sec zip my value time = 2.72's, unzip length = 0.247'GB zip my value throughput = 90.913'MB/sec zip pipeline throughput = 90.913'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1729 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 20.9715 value = 1.4279 dict = 1.00 all = 1.4863 } Zip/UnZip{ index = 0.0477 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 65.483947 GB avg = 0.180 KB (by entry num) total key len = 7.055748 GB avg = 0.019 KB total ukey len = 3.323003 GB avg = 0.008 KB total ukey num = 0.406020455 Billion total entry num = 0.362898463 Billion write speed all = 2.872212400 MB/sec (with seq num) write speed all = 2.757260555 MB/sec (without seq num) 2024/08/20-00:34:57.091971 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622287: 1342178 keys, 180603181 bytes, temperature: kWarm 2024/08/20-00:34:57.092041 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085297092007, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622287, "file_size": 180603181, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189414097160, "largest_seqno": 189634997862, "table_properties": {"data_size": 172947904, "index_size": 512000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085291, "oldest_key_time": 0, "file_creation_time": 1724085291, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622287, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:34:57.156460 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622288.sst) /dev/shm/ToplingTemp/Topling-78558-OeLkGm 2024/08/20-00:35:00.197693 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622288.sst): first pass time = 3.04's, 88.266'MB/sec 2024/08/20-00:35:02.901309 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622288.sst): old prealloc_size = 295279001, real_size = 173805504 2024/08/20-00:35:02.901389 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622288.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:35:03.967580 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622288.sst): second pass time = 2.70's, 91.400'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 169.388'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 16547.584'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.386'MB/sec write SST data time = 0.20's, 919.696'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 54354.710'MB/sec zip my value time = 2.70's, unzip length = 0.247'GB zip my value throughput = 91.400'MB/sec zip pipeline throughput = 91.400'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 20.9899 value = 1.4279 dict = 1.00 all = 1.4862 } Zip/UnZip{ index = 0.0476 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 65.740303 GB avg = 0.180 KB (by entry num) total key len = 7.077223 GB avg = 0.019 KB total ukey len = 3.333740 GB avg = 0.008 KB total ukey num = 0.407362633 Billion total entry num = 0.364240641 Billion write speed all = 2.882405193 MB/sec (with seq num) write speed all = 2.767060530 MB/sec (without seq num) 2024/08/20-00:35:04.166998 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622288: 1342178 keys, 180613037 bytes, temperature: kWarm 2024/08/20-00:35:04.167066 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085304167033, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622288, "file_size": 180613037, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189414096942, "largest_seqno": 189634997810, "table_properties": {"data_size": 172958192, "index_size": 511552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085297, "oldest_key_time": 0, "file_creation_time": 1724085297, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622288, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:04.232966 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622289.sst) /dev/shm/ToplingTemp/Topling-78558-ubz1Ro 2024/08/20-00:35:07.011259 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622289.sst): first pass time = 2.78's, 96.620'MB/sec 2024/08/20-00:35:09.768621 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622289.sst): old prealloc_size = 295279001, real_size = 173800608 2024/08/20-00:35:09.768711 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622289.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:35:10.831563 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622289.sst): second pass time = 2.76's, 89.625'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 169.915'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15848.809'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.387'MB/sec write SST data time = 0.20's, 919.950'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 54279.877'MB/sec zip my value time = 2.76's, unzip length = 0.247'GB zip my value throughput = 89.625'MB/sec zip pipeline throughput = 89.625'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 20.9899 value = 1.4279 dict = 1.00 all = 1.4863 } Zip/UnZip{ index = 0.0476 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 65.996659 GB avg = 0.181 KB (by entry num) total key len = 7.098698 GB avg = 0.019 KB total ukey len = 3.344477 GB avg = 0.008 KB total ukey num = 0.408704811 Billion total entry num = 0.365582819 Billion write speed all = 2.892616901 MB/sec (with seq num) write speed all = 2.776878655 MB/sec (without seq num) 2024/08/20-00:35:11.031611 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622289: 1342178 keys, 180608109 bytes, temperature: kWarm 2024/08/20-00:35:11.031682 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085311031648, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622289, "file_size": 180608109, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189414096946, "largest_seqno": 189634997858, "table_properties": {"data_size": 172953296, "index_size": 511552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085304, "oldest_key_time": 0, "file_creation_time": 1724085304, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622289, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:11.099047 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622290.sst) /dev/shm/ToplingTemp/Topling-78558-5x4dcn 2024/08/20-00:35:14.136136 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622290.sst): first pass time = 3.04's, 88.387'MB/sec 2024/08/20-00:35:16.855955 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622290.sst): old prealloc_size = 295279001, real_size = 173801864 2024/08/20-00:35:16.856033 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622290.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:35:17.928679 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622290.sst): second pass time = 2.72's, 90.860'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 168.364'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15220.336'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.383'MB/sec write SST data time = 0.20's, 920.900'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 37190.239'MB/sec zip my value time = 2.72's, unzip length = 0.247'GB zip my value throughput = 90.860'MB/sec zip pipeline throughput = 90.860'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 20.9742 value = 1.4279 dict = 1.00 all = 1.4863 } Zip/UnZip{ index = 0.0477 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 66.253015 GB avg = 0.181 KB (by entry num) total key len = 7.120173 GB avg = 0.019 KB total ukey len = 3.355215 GB avg = 0.008 KB total ukey num = 0.410046989 Billion total entry num = 0.366924997 Billion write speed all = 2.902796289 MB/sec (with seq num) write speed all = 2.786665745 MB/sec (without seq num) 2024/08/20-00:35:18.136256 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622290: 1342178 keys, 180609389 bytes, temperature: kWarm 2024/08/20-00:35:18.136327 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085318136293, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622290, "file_size": 180609389, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189414096943, "largest_seqno": 189634997865, "table_properties": {"data_size": 172954192, "index_size": 511936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085311, "oldest_key_time": 0, "file_creation_time": 1724085311, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622290, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:18.205376 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622291.sst) /dev/shm/ToplingTemp/Topling-78558-K8thco 2024/08/20-00:35:21.243085 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622291.sst): first pass time = 3.04's, 88.369'MB/sec 2024/08/20-00:35:23.948266 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622291.sst): old prealloc_size = 295279001, real_size = 173798560 2024/08/20-00:35:23.948349 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622291.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:35:25.012882 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622291.sst): second pass time = 2.70's, 91.347'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 169.645'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 16021.605'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.387'MB/sec write SST data time = 0.20's, 911.342'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 61152.278'MB/sec zip my value time = 2.70's, unzip length = 0.247'GB zip my value throughput = 91.347'MB/sec zip pipeline throughput = 91.347'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.0452 value = 1.4279 dict = 1.00 all = 1.4863 } Zip/UnZip{ index = 0.0475 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 66.509371 GB avg = 0.181 KB (by entry num) total key len = 7.141647 GB avg = 0.019 KB total ukey len = 3.365952 GB avg = 0.008 KB total ukey num = 0.411389167 Billion total entry num = 0.368267175 Billion write speed all = 2.912971449 MB/sec (with seq num) write speed all = 2.796448767 MB/sec (without seq num) 2024/08/20-00:35:25.213715 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622291: 1342178 keys, 180606125 bytes, temperature: kWarm 2024/08/20-00:35:25.213783 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085325213751, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622291, "file_size": 180606125, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412283398, "largest_seqno": 189634997864, "table_properties": {"data_size": 172952608, "index_size": 510208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085318, "oldest_key_time": 0, "file_creation_time": 1724085318, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622291, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:25.279295 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622292.sst) /dev/shm/ToplingTemp/Topling-78558-4seG5m 2024/08/20-00:35:26.779314 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622292.sst): first pass time = 1.50's, 178.961'MB/sec 2024/08/20-00:35:29.507523 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622292.sst): old prealloc_size = 295279001, real_size = 173789072 2024/08/20-00:35:29.507588 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622292.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1729 GB avg-val = 184.00 avg-zval = 128.85 2024/08/20-00:35:30.572823 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622292.sst): second pass time = 2.73's, 90.582'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 169.528'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15827.421'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.386'MB/sec write SST data time = 0.20's, 917.144'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 58977.489'MB/sec zip my value time = 2.73's, unzip length = 0.247'GB zip my value throughput = 90.582'MB/sec zip pipeline throughput = 90.582'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.85 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1729 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.1487 value = 1.4280 dict = 1.00 all = 1.4864 } Zip/UnZip{ index = 0.0473 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 66.765727 GB avg = 0.181 KB (by entry num) total key len = 7.163122 GB avg = 0.019 KB total ukey len = 3.376690 GB avg = 0.008 KB total ukey num = 0.412731345 Billion total entry num = 0.369609353 Billion write speed all = 2.923317097 MB/sec (with seq num) write speed all = 2.806395451 MB/sec (without seq num) 2024/08/20-00:35:30.774472 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622292: 1342178 keys, 180596589 bytes, temperature: kWarm 2024/08/20-00:35:30.774546 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085330774510, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622292, "file_size": 180596589, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410457285, "largest_seqno": 189634997866, "table_properties": {"data_size": 172945600, "index_size": 507712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085325, "oldest_key_time": 0, "file_creation_time": 1724085325, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622292, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:30.841115 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622293.sst) /dev/shm/ToplingTemp/Topling-78558-KSoHoo 2024/08/20-00:35:32.817975 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622293.sst): first pass time = 1.98's, 16.612'MB/sec 2024/08/20-00:35:33.075682 140500271638080 [top_zip_table_builder.cc:2718] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622293.sst): part-0: reject ZipStore ratio = 0.818 2024/08/20-00:35:33.145094 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622293.sst): old prealloc_size = 295279001, real_size = 30315032 2024/08/20-00:35:33.145144 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622293.sst): index type = UintIndex_IL_256_32 , store type = MixedLenBlobStore usrkeys = 164194 prefix = 4 raw-key = 0.0007 GB zip-key = 0.0001 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.0302 GB zip-val = 0.0302 GB avg-val = 184.00 avg-zval = 184.00 2024/08/20-00:35:33.199372 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622293.sst): second pass time = 0.26's, 117.561'MB/sec, value only(92.0% of KV) wait indexing time = 0.07's, remap KeyValue time = 0.05's, 568.385'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1964.111'MB/sec (index lex order gen) rebuild zvType time = 0.03's, 1.331'MB/sec write SST data time = 0.02's, 1317.496'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.26's, unzip length = 0.030'GB zip my value throughput = 117.561'MB/sec zip pipeline throughput = 117.561'MB/sec entries = 164194 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 184.00 usrkeys = 164194 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 184.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 164194 bytes = 574688 } seq expand size = 1149358 multi value expand size = 0 cnt WriteAppend = 3695 UnZipSize{ index = 0.0013 GB value = 0.0302 GB dict = 0.00 MB all = 0.0328 GB } __ZipSize{ index = 0.0001 GB value = 0.0302 GB dict = 0.00 MB all = 0.0308 GB } UnZip/Zip{ index = 21.2027 value = 1.0000 dict = 0.00 all = 1.0645 } Zip/UnZip{ index = 0.0472 value = 1.0000 dict = 0.00 all = 0.9394 } ---------------------------- total value len = 66.797088 GB avg = 0.181 KB (by entry num) total key len = 7.165749 GB avg = 0.019 KB total ukey len = 3.378003 GB avg = 0.008 KB total ukey num = 0.412895539 Billion total entry num = 0.369773547 Billion write speed all = 2.924357344 MB/sec (with seq num) write speed all = 2.807395904 MB/sec (without seq num) 2024/08/20-00:35:33.242281 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622293: 164194 keys, 30849706 bytes, temperature: kWarm 2024/08/20-00:35:33.242353 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085333242318, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622293, "file_size": 30849706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189410455675, "largest_seqno": 189634997803, "table_properties": {"data_size": 30211888, "index_size": 61952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2627104, "raw_average_key_size": 16, "raw_value_size": 30211696, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 164194, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-MixedLenBlobStore", "creation_time": 1724085330, "oldest_key_time": 0, "file_creation_time": 1724085330, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622293, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:33.268948 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622294.sst) /dev/shm/ToplingTemp/Topling-78558-qGpoRm 2024/08/20-00:35:34.496620 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622294.sst): first pass time = 1.23's, 218.663'MB/sec 2024/08/20-00:35:37.226277 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622294.sst): old prealloc_size = 295279001, real_size = 173779376 2024/08/20-00:35:37.226340 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622294.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1729 GB avg-val = 184.00 avg-zval = 128.85 2024/08/20-00:35:38.289355 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622294.sst): second pass time = 2.73's, 90.536'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 169.873'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 14737.708'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.386'MB/sec write SST data time = 0.19's, 932.498'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 55046.276'MB/sec zip my value time = 2.73's, unzip length = 0.247'GB zip my value throughput = 90.536'MB/sec zip pipeline throughput = 90.536'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.85 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1729 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.1087 value = 1.4281 dict = 1.00 all = 1.4865 } Zip/UnZip{ index = 0.0474 value = 0.7003 dict = 1.00 all = 0.6727 } ---------------------------- total value len = 67.053444 GB avg = 0.181 KB (by entry num) total key len = 7.187224 GB avg = 0.019 KB total ukey len = 3.388741 GB avg = 0.008 KB total ukey num = 0.414237717 Billion total entry num = 0.371115725 Billion write speed all = 2.934751660 MB/sec (with seq num) write speed all = 2.817389301 MB/sec (without seq num) 2024/08/20-00:35:38.487966 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622294: 1342178 keys, 180586925 bytes, temperature: kWarm 2024/08/20-00:35:38.488036 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085338488001, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622294, "file_size": 180586925, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412282327, "largest_seqno": 189634998181, "table_properties": {"data_size": 172934944, "index_size": 508672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085333, "oldest_key_time": 0, "file_creation_time": 1724085333, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622294, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:38.560203 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622295.sst) /dev/shm/ToplingTemp/Topling-78558-4yjGyp 2024/08/20-00:35:41.437798 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622295.sst): first pass time = 2.88's, 93.286'MB/sec 2024/08/20-00:35:44.137365 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622295.sst): old prealloc_size = 295279001, real_size = 173801432 2024/08/20-00:35:44.137466 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622295.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1730 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:35:45.200959 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622295.sst): second pass time = 2.70's, 91.544'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 169.814'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 14330.000'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.386'MB/sec write SST data time = 0.19's, 929.719'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 56058.709'MB/sec zip my value time = 2.70's, unzip length = 0.247'GB zip my value throughput = 91.544'MB/sec zip pipeline throughput = 91.544'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1730 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.1087 value = 1.4279 dict = 1.00 all = 1.4863 } Zip/UnZip{ index = 0.0474 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 67.309800 GB avg = 0.181 KB (by entry num) total key len = 7.208699 GB avg = 0.019 KB total ukey len = 3.399478 GB avg = 0.008 KB total ukey num = 0.415579895 Billion total entry num = 0.372457903 Billion write speed all = 2.944929777 MB/sec (with seq num) write speed all = 2.827175137 MB/sec (without seq num) 2024/08/20-00:35:45.401173 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622295: 1342178 keys, 180609005 bytes, temperature: kWarm 2024/08/20-00:35:45.401241 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085345401209, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622295, "file_size": 180609005, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412282139, "largest_seqno": 189634998184, "table_properties": {"data_size": 172957040, "index_size": 508672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085338, "oldest_key_time": 0, "file_creation_time": 1724085338, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622295, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:45.471953 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622296.sst) /dev/shm/ToplingTemp/Topling-78558-qNjd0m 2024/08/20-00:35:48.637957 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622296.sst): first pass time = 3.17's, 84.788'MB/sec 2024/08/20-00:35:51.356419 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622296.sst): old prealloc_size = 295279001, real_size = 173794064 2024/08/20-00:35:51.356499 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622296.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0005 GB avg-key = 4.00 avg-zkey = 0.38 raw-val = 0.2470 GB zip-val = 0.1729 GB avg-val = 184.00 avg-zval = 128.86 2024/08/20-00:35:52.419139 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622296.sst): second pass time = 2.72's, 90.902'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 169.944'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 16131.896'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.388'MB/sec write SST data time = 0.20's, 913.663'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 49798.114'MB/sec zip my value time = 2.72's, unzip length = 0.247'GB zip my value throughput = 90.902'MB/sec zip pipeline throughput = 90.902'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.38 avg-val = 184.00 avg-zval = 128.86 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.38 avg-val = 191.00 avg-zval = 128.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0005 GB value = 0.1729 GB dict = 2.44 MB all = 0.1806 GB } UnZip/Zip{ index = 21.1034 value = 1.4279 dict = 1.00 all = 1.4863 } Zip/UnZip{ index = 0.0474 value = 0.7003 dict = 1.00 all = 0.6728 } ---------------------------- total value len = 67.566156 GB avg = 0.181 KB (by entry num) total key len = 7.230174 GB avg = 0.019 KB total ukey len = 3.410216 GB avg = 0.008 KB total ukey num = 0.416922073 Billion total entry num = 0.373800081 Billion write speed all = 2.955066539 MB/sec (with seq num) write speed all = 2.836921264 MB/sec (without seq num) 2024/08/20-00:35:52.617751 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622296: 1342178 keys, 180601645 bytes, temperature: kWarm 2024/08/20-00:35:52.617820 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085352617786, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622296, "file_size": 180601645, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412282138, "largest_seqno": 189634998176, "table_properties": {"data_size": 172949552, "index_size": 508800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085345, "oldest_key_time": 0, "file_creation_time": 1724085345, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622296, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:52.694734 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622297.sst) /dev/shm/ToplingTemp/Topling-78558-um2xcm 2024/08/20-00:35:53.938394 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622297.sst): first pass time = 1.24's, 215.853'MB/sec 2024/08/20-00:35:56.650959 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622297.sst): old prealloc_size = 295279001, real_size = 173869536 2024/08/20-00:35:56.651037 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622297.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1342178 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0006 GB avg-key = 4.00 avg-zkey = 0.44 raw-val = 0.2470 GB zip-val = 0.1729 GB avg-val = 184.00 avg-zval = 128.85 2024/08/20-00:35:57.716556 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622297.sst): second pass time = 2.71's, 91.101'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 169.552'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 15066.629'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.387'MB/sec write SST data time = 0.20's, 911.851'MB/sec dict compress time = 0.00's, 2444440.000'MB/sec z-dict build time = 0.00's, sample length = 2.444'MB, throughput = 60262.801'MB/sec zip my value time = 2.71's, unzip length = 0.247'GB zip my value throughput = 91.101'MB/sec zip pipeline throughput = 91.101'MB/sec entries = 1342178 avg-key = 16.00 avg-zkey = 0.44 avg-val = 184.00 avg-zval = 128.85 usrkeys = 1342178 avg-key = 8.00 avg-zkey = 0.44 avg-val = 191.00 avg-zval = 128.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1342178 bytes = 4697632 } seq expand size = 9395246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0107 GB value = 0.2470 GB dict = 2.44 MB all = 0.2684 GB } __ZipSize{ index = 0.0006 GB value = 0.1729 GB dict = 2.44 MB all = 0.1807 GB } UnZip/Zip{ index = 18.2421 value = 1.4280 dict = 1.00 all = 1.4857 } Zip/UnZip{ index = 0.0548 value = 0.7003 dict = 1.00 all = 0.6731 } ---------------------------- total value len = 67.822512 GB avg = 0.181 KB (by entry num) total key len = 7.251649 GB avg = 0.019 KB total ukey len = 3.420953 GB avg = 0.008 KB total ukey num = 0.418264251 Billion total entry num = 0.375142259 Billion write speed all = 2.965422491 MB/sec (with seq num) write speed all = 2.846877811 MB/sec (without seq num) 2024/08/20-00:35:57.915333 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622297: 1342178 keys, 180677101 bytes, temperature: kWarm 2024/08/20-00:35:57.915408 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085357915370, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622297, "file_size": 180677101, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412282132, "largest_seqno": 189634997161, "table_properties": {"data_size": 172945184, "index_size": 588608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21474848, "raw_average_key_size": 16, "raw_value_size": 246960752, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1342178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1724085352, "oldest_key_time": 0, "file_creation_time": 1724085352, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622297, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:57.979228 140500271638080 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622298.sst) /dev/shm/ToplingTemp/Topling-78558-qR8eTn 2024/08/20-00:35:58.119457 140500271638080 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622298.sst): first pass time = 0.14's, 224.669'MB/sec 2024/08/20-00:35:58.375804 140500271638080 [top_zip_table_builder.cc:2718] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622298.sst): part-0: reject ZipStore ratio = 0.821 2024/08/20-00:35:58.443700 140500271638080 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622298.sst): old prealloc_size = 295279001, real_size = 29088064 2024/08/20-00:35:58.443750 140500271638080 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622298.sst): index type = UintIndex_IL_256_32 , store type = MixedLenBlobStore usrkeys = 157481 prefix = 4 raw-key = 0.0006 GB zip-key = 0.0001 GB avg-key = 4.00 avg-zkey = 0.46 raw-val = 0.0290 GB zip-val = 0.0290 GB avg-val = 184.00 avg-zval = 184.00 2024/08/20-00:35:58.496844 140500271638080 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00/14622298.sst): second pass time = 0.26's, 113.342'MB/sec, value only(92.0% of KV) wait indexing time = 0.07's, remap KeyValue time = 0.05's, 557.030'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2233.947'MB/sec (index lex order gen) rebuild zvType time = 0.03's, 1.277'MB/sec write SST data time = 0.02's, 1328.072'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.26's, unzip length = 0.029'GB zip my value throughput = 113.342'MB/sec zip pipeline throughput = 113.342'MB/sec entries = 157481 avg-key = 16.00 avg-zkey = 0.46 avg-val = 184.00 avg-zval = 184.00 usrkeys = 157481 avg-key = 8.00 avg-zkey = 0.46 avg-val = 191.00 avg-zval = 184.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 157481 bytes = 551200 } seq expand size = 1102367 multi value expand size = 0 cnt WriteAppend = 3544 UnZipSize{ index = 0.0013 GB value = 0.0290 GB dict = 0.00 MB all = 0.0315 GB } __ZipSize{ index = 0.0001 GB value = 0.0290 GB dict = 0.00 MB all = 0.0296 GB } UnZip/Zip{ index = 17.5447 value = 1.0000 dict = 0.00 all = 1.0640 } Zip/UnZip{ index = 0.0570 value = 1.0000 dict = 0.00 all = 0.9398 } ---------------------------- total value len = 67.852590 GB avg = 0.181 KB (by entry num) total key len = 7.254168 GB avg = 0.019 KB total ukey len = 3.422213 GB avg = 0.008 KB total ukey num = 0.418421732 Billion total entry num = 0.375299740 Billion write speed all = 2.966618696 MB/sec (with seq num) write speed all = 2.848027907 MB/sec (without seq num) 2024/08/20-00:35:58.533675 140500271638080 [compaction_job.cc:1944] [default] [JOB 2410] Generated table #14622298: 157481 keys, 29600874 bytes, temperature: kWarm 2024/08/20-00:35:58.533744 140500271638080 EVENT_LOG_v1 {"time_micros": 1724085358533712, "cf_name": "default", "job": 2410, "event": "table_file_creation", "file_number": 14622298, "file_size": 29600874, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 189412284733, "largest_seqno": 189576953200, "table_properties": {"data_size": 28976704, "index_size": 71808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2519696, "raw_average_key_size": 16, "raw_value_size": 28976504, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 157481, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-MixedLenBlobStore", "creation_time": 1724085357, "oldest_key_time": 0, "file_creation_time": 1724085357, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "9474K4QZ2QA6TIYF9LEW", "orig_file_number": 14622298, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/08/20-00:35:58.548561 140500271638080 [compaction_job.cc:699] job-02410: subcompact[0], size: 3.939844 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02410/att-00 [14622274,14622275,14622276,14622277,14622278,14622279,14622280,14622281,14622282,14622283,14622284,14622285,14622286,14622287,14622288,14622289,14622290,14622291,14622292,14622293,14622294,14622295,14622296,14622297,14622298] 2024/08/20-00:35:59.039027 140500271638080 (Original Log Time 2024/08/20-00:33:21.487514) EVENT_LOG_v1 {"time_micros": 1724085201487328, "job": 2410, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L3": [14622118, 14622119, 14622121, 14622122, 14622211, 14622212, 14622214, 14622215, 14622216, 14622128, 14622129, 14622130, 14622131, 14622132, 14622133, 14622134, 14622135], "files_L4": [14622271, 14620643, 14622254, 14621777, 14620022, 14621663, 14621664, 14620023, 14622257, 14620428, 14620429, 14619778, 14621662, 14621033, 14620532, 14620943, 14621132, 14621133, 14620641, 14620642, 14619796], "score": -1, "input_data_size": 4335853127, "oldest_snapshot_seqno": 189641431181} 2024/08/20-00:35:59.039031 140500271638080 (Original Log Time 2024/08/20-00:35:59.037177) [compaction_job.cc:2018] [default] [JOB 2410] Compacted 17@3 + 21@4 files to L4 => 3939843531 bytes 2024/08/20-00:35:59.039033 140500271638080 (Original Log Time 2024/08/20-00:35:59.038943) [compaction_job.cc:1220] [default] compacted to: files[0 0 0 0 25 0 0] max score 0.01, MB/sec: 27.6 rd, 25.1 wr, level 4, files in(17, 21) out(25 +0 blob) MB in(1344.4, 2790.6 +0.0 blob) out(3757.3 +0.0 blob), read-write-amplify(5.9) write-amplify(2.8) OK, records in: 31227197, records dropped: 2139026 output_compression: Snappy 2024/08/20-00:35:59.039036 140500271638080 (Original Log Time 2024/08/20-00:35:59.038998) EVENT_LOG_v1 {"time_micros": 1724085359038971, "cf": "default", "job": 2410, "event": "compaction_finished", "compaction_time_micros": 157061054, "compaction_time_cpu_micros": 55967500, "output_level": 4, "num_output_files": 25, "total_output_size": 3939843531, "num_input_records": 31227197, "num_output_records": 29088171, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 2327216262, "file_range_sync_nanos": 0, "file_fsync_nanos": 17735, "file_prepare_write_nanos": 316823, "lsm_state": [0, 0, 0, 0, 25, 0, 0]} 2024/08/20-00:35:59.178892 140500271638080 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"a007a635-5c0a-4b4a-be96-b9c695cb016b","dbStarts":1724082090,"starts":1724085201,"executesMs":157704,"compactionJobId":2410,"attempt":0,"compactionInputRawBytes":6245439400,"compactionInputZipBytes":4335853127,"compactionOutputRawBytes":0,"compactionOutputZipBytes":3939843531,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-4bc384d02be12ccc1c1e46bf82ff9380-fbd7fef0268f1951-00"}}, response = 2024/08/20-00:35:59.179001 140500271638080 INFO dcompact_worker.cpp:1456: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-08-19T23.41.30/.rocksdb/job-02410/att-00: olev 4, work 157.704 s, result 6.160 ms, install 1.889 ms, input{raw 5.817 GiB zip 4.038 GiB}