2024/11/23-14:14:13.910541 139957458552384 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00: fork to child time = 0.007495 sec 2024/11/23-14:14:13.911273 139957458552384 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00 2024/11/23-14:14:13.918264 139957458552384 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00 2024/11/23-14:14:13.919177 139957458552384 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:14:13.919439 139957458552384 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:14:13.976654 139957458552384 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27430/att-00/MANIFEST-206347 2024/11/23-14:14:14.010346 139957458552384 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27430/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 262580, last_sequence is 4012741837, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 262359 2024/11/23-14:14:14.010353 139957458552384 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:14:14.010457 139957458552384 [version_set.cc:5721] Creating manifest 262580 2024/11/23-14:14:14.053940 139957458552384 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:14:14.074663 139957458552384 [compaction_job.cc:2437] [default] [JOB 27430] Compacting 1@2 + 14@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-14:14:14.074676 139957458552384 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20546 Base level 2, inputs: [262509(21MB)], [262563(35MB) 262564(8034KB) 262469(7527KB) 262430(35MB) 262431(34MB) 262432(35MB) 262433(34MB) 262434(35MB) 262435(33MB) 262436(33MB) 262437(33MB) 262438(34MB) 262439(36MB) 262565(35MB)] 2024/11/23-14:14:14.080530 139957458552384 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:14:14.080561 139957458552384 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:14:14.080580 139957458552384 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:14:14.080586 139957458552384 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:14:14.080596 139957458552384 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:14:14.670425 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262581.sst) /dev/shm/ToplingTemp/Topling-69172-oSQGQD 2024/11/23-14:14:15.217080 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262581.sst): first pass time = 0.55's, 245.545'MB/sec 2024/11/23-14:14:16.525936 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262581.sst): old prealloc_size = 147639500, real_size = 32859200 2024/11/23-14:14:16.526543 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262581.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931642 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.07 avg-zval = 33.11 2024/11/23-14:14:16.527376 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262581.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:17.381793 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262581.sst): second pass time = 1.31's, 82.681'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.334'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3046.167'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 301.298'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33325.274'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.681'MB/sec zip pipeline throughput = 82.681'MB/sec entries = 931642 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.07 avg-zval = 33.11 usrkeys = 931642 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.07 avg-zval = 33.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931642 bytes = 3377216 } seq expand size = 6521494 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4459 value = 3.5060 dict = 1.00 all = 3.6190 } Zip/UnZip{ index = 0.0957 value = 0.2852 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 8.963273 GB avg = 0.168 KB (by entry num) total key len = 1.466907 GB avg = 0.028 KB total ukey len = 1.016511 GB avg = 0.015 KB total ukey num = 0.066482077 Billion total entry num = 0.053289806 Billion write speed all = 0.544794420 MB/sec (with seq num) write speed all = 0.522526740 MB/sec (without seq num) 2024/11/23-14:14:17.428828 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262581: 931642 keys, 37087394 bytes, temperature: kWarm 2024/11/23-14:14:17.428926 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342457428865, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262581, "file_size": 37087394, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 4012410499, "table_properties": {"data_size": 30842352, "index_size": 1783744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26085976, "raw_average_key_size": 28, "raw_value_size": 108131888, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931642, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342454, "oldest_key_time": 0, "file_creation_time": 1732342454, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262581, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:17.468741 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262582.sst) /dev/shm/ToplingTemp/Topling-69172-4OuE3C 2024/11/23-14:14:18.682449 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262582.sst): first pass time = 1.21's, 110.589'MB/sec 2024/11/23-14:14:19.972463 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262582.sst): old prealloc_size = 147639500, real_size = 32430960 2024/11/23-14:14:19.973059 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262582.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934479 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.63 avg-zval = 32.54 2024/11/23-14:14:19.973883 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262582.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:20.832391 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262582.sst): second pass time = 1.29's, 83.813'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.779'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3108.301'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.318'MB/sec write SST data time = 0.12's, 297.466'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36113.214'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.813'MB/sec zip pipeline throughput = 83.813'MB/sec entries = 934479 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.63 avg-zval = 32.54 usrkeys = 934479 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.63 avg-zval = 32.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934479 bytes = 3504304 } seq expand size = 6541353 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4470 value = 3.5534 dict = 1.00 all = 3.6486 } Zip/UnZip{ index = 0.0957 value = 0.2814 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 9.077867 GB avg = 0.167 KB (by entry num) total key len = 1.493072 GB avg = 0.028 KB total ukey len = 1.035200 GB avg = 0.015 KB total ukey num = 0.067416556 Billion total entry num = 0.054224285 Billion write speed all = 0.552047127 MB/sec (with seq num) write speed all = 0.529393049 MB/sec (without seq num) 2024/11/23-14:14:20.877820 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262582: 934479 keys, 36785632 bytes, temperature: kWarm 2024/11/23-14:14:20.877888 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342460877853, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262582, "file_size": 36785632, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012411092, "table_properties": {"data_size": 30408160, "index_size": 1788992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26165412, "raw_average_key_size": 28, "raw_value_size": 108052414, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934479, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342457, "oldest_key_time": 0, "file_creation_time": 1732342457, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262582, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:20.925823 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262583.sst) /dev/shm/ToplingTemp/Topling-69172-lwBvvF 2024/11/23-14:14:21.940484 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262583.sst): first pass time = 1.01's, 132.285'MB/sec 2024/11/23-14:14:23.237158 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262583.sst): old prealloc_size = 147639500, real_size = 31829376 2024/11/23-14:14:23.237746 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262583.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933828 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0298 GB avg-val = 115.73 avg-zval = 31.93 2024/11/23-14:14:23.238556 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262583.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:24.081275 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262583.sst): second pass time = 1.30's, 83.394'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.726'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3132.392'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.12's, 289.969'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34115.320'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.394'MB/sec zip pipeline throughput = 83.394'MB/sec entries = 933828 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.73 avg-zval = 31.93 usrkeys = 933828 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.73 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933828 bytes = 3385136 } seq expand size = 6536796 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5142 value = 3.6242 dict = 1.00 all = 3.7216 } Zip/UnZip{ index = 0.0951 value = 0.2759 dict = 1.00 all = 0.2687 } ---------------------------- total value len = 9.192474 GB avg = 0.167 KB (by entry num) total key len = 1.519220 GB avg = 0.028 KB total ukey len = 1.053877 GB avg = 0.015 KB total ukey num = 0.068350384 Billion total entry num = 0.055158113 Billion write speed all = 0.559302873 MB/sec (with seq num) write speed all = 0.536262565 MB/sec (without seq num) 2024/11/23-14:14:24.125884 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262583: 933828 keys, 36064970 bytes, temperature: kWarm 2024/11/23-14:14:24.125954 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342464125919, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262583, "file_size": 36064970, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 4012411299, "table_properties": {"data_size": 29819440, "index_size": 1776320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26147184, "raw_average_key_size": 28, "raw_value_size": 108070640, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933828, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342460, "oldest_key_time": 0, "file_creation_time": 1732342460, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262583, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:24.160778 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262584.sst) /dev/shm/ToplingTemp/Topling-69172-Oz1k7G 2024/11/23-14:14:25.183109 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262584.sst): first pass time = 1.02's, 131.291'MB/sec 2024/11/23-14:14:26.449867 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262584.sst): old prealloc_size = 147639500, real_size = 31614000 2024/11/23-14:14:26.450424 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262584.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933922 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 115.71 avg-zval = 31.70 2024/11/23-14:14:26.451235 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262584.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:27.287753 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262584.sst): second pass time = 1.27's, 85.365'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.648'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3319.569'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.12's, 288.862'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34930.680'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.365'MB/sec zip pipeline throughput = 85.365'MB/sec entries = 933922 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.71 avg-zval = 31.70 usrkeys = 933922 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.71 avg-zval = 31.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933922 bytes = 3268736 } seq expand size = 6537454 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5134 value = 3.6505 dict = 1.00 all = 3.7561 } Zip/UnZip{ index = 0.0951 value = 0.2739 dict = 1.00 all = 0.2662 } ---------------------------- total value len = 9.307080 GB avg = 0.166 KB (by entry num) total key len = 1.545369 GB avg = 0.028 KB total ukey len = 1.072555 GB avg = 0.015 KB total ukey num = 0.069284306 Billion total entry num = 0.056092035 Billion write speed all = 0.566557449 MB/sec (with seq num) write speed all = 0.543130951 MB/sec (without seq num) 2024/11/23-14:14:27.331912 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262584: 933922 keys, 35733040 bytes, temperature: kWarm 2024/11/23-14:14:27.332005 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342467331948, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262584, "file_size": 35733040, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012411538, "table_properties": {"data_size": 29603680, "index_size": 1776640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26149816, "raw_average_key_size": 28, "raw_value_size": 108068048, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933922, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342464, "oldest_key_time": 0, "file_creation_time": 1732342464, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262584, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:27.370468 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262585.sst) /dev/shm/ToplingTemp/Topling-69172-aAvRgH 2024/11/23-14:14:28.391038 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262585.sst): first pass time = 1.02's, 131.517'MB/sec 2024/11/23-14:14:29.674126 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262585.sst): old prealloc_size = 147639500, real_size = 30570432 2024/11/23-14:14:29.674673 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262585.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931940 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0286 GB avg-val = 116.02 avg-zval = 30.65 2024/11/23-14:14:29.675470 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262585.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:30.498634 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262585.sst): second pass time = 1.28's, 84.326'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.039'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3363.481'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 267.688'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31607.682'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.326'MB/sec zip pipeline throughput = 84.326'MB/sec entries = 931940 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.02 avg-zval = 30.65 usrkeys = 931940 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.65 avg-zval = 30.65 TagRS{ kinds = RS_Key0_TagN bytes = 145720 } TagArray{ size = 882674 bytes = 3089376 } seq expand size = 6178718 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.5138 value = 3.7852 dict = 1.00 all = 3.8724 } Zip/UnZip{ index = 0.0951 value = 0.2642 dict = 1.00 all = 0.2582 } ---------------------------- total value len = 9.421382 GB avg = 0.165 KB (by entry num) total key len = 1.571464 GB avg = 0.028 KB total ukey len = 1.091194 GB avg = 0.016 KB total ukey num = 0.070216246 Billion total entry num = 0.057023975 Billion write speed all = 0.573790737 MB/sec (with seq num) write speed all = 0.549979011 MB/sec (without seq num) 2024/11/23-14:14:30.541578 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262585: 931940 keys, 34660204 bytes, temperature: kWarm 2024/11/23-14:14:30.541647 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342470541611, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262585, "file_size": 34660204, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012411589, "table_properties": {"data_size": 28564464, "index_size": 1772800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26094320, "raw_average_key_size": 28, "raw_value_size": 108123500, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931940, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342467, "oldest_key_time": 0, "file_creation_time": 1732342467, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262585, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:30.578371 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262586.sst) /dev/shm/ToplingTemp/Topling-69172-o7963C 2024/11/23-14:14:31.611507 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262586.sst): first pass time = 1.03's, 129.918'MB/sec 2024/11/23-14:14:32.998863 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262586.sst): old prealloc_size = 147639500, real_size = 34943280 2024/11/23-14:14:32.999485 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262586.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081522 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.10 avg-zval = 30.16 2024/11/23-14:14:32.999526 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262586.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:33.675628 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262586.sst): second pass time = 1.39's, 74.995'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.68's, 52.776'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3412.594'MB/sec (index lex order gen) rebuild zvType time = 0.54's, 0.501'MB/sec write SST data time = 0.14's, 261.817'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.032'MB, throughput = 34002.835'MB/sec zip my value time = 1.39's, unzip length = 0.104'GB zip my value throughput = 74.995'MB/sec zip pipeline throughput = 74.995'MB/sec entries = 1081522 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.10 avg-zval = 30.16 usrkeys = 1081522 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.10 avg-zval = 30.16 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.03 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5164 value = 3.1866 dict = 1.00 all = 3.7580 } Zip/UnZip{ index = 0.0951 value = 0.3138 dict = 1.00 all = 0.2661 } ---------------------------- total value len = 9.525317 GB avg = 0.164 KB (by entry num) total key len = 1.601746 GB avg = 0.028 KB total ukey len = 1.112825 GB avg = 0.016 KB total ukey num = 0.071297768 Billion total entry num = 0.058105497 Billion write speed all = 0.580700175 MB/sec (with seq num) write speed all = 0.556440857 MB/sec (without seq num) 2024/11/23-14:14:33.719524 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262586: 1081522 keys, 35715424 bytes, temperature: kWarm 2024/11/23-14:14:33.719592 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342473719557, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262586, "file_size": 35715424, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32615904, "index_size": 2056832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30282616, "raw_average_key_size": 28, "raw_value_size": 103935245, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081522, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342470, "oldest_key_time": 0, "file_creation_time": 1732342470, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262586, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:33.766751 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262587.sst) /dev/shm/ToplingTemp/Topling-69172-3fGxDD 2024/11/23-14:14:34.747905 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262587.sst): first pass time = 0.98's, 136.800'MB/sec 2024/11/23-14:14:36.053080 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262587.sst): old prealloc_size = 147639500, real_size = 34189792 2024/11/23-14:14:36.053632 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262587.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921893 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0322 GB avg-val = 117.59 avg-zval = 34.92 2024/11/23-14:14:36.054277 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262587.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:36.866826 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262587.sst): second pass time = 1.30's, 83.108'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 46.315'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3333.287'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 301.303'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33805.429'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.108'MB/sec zip pipeline throughput = 83.108'MB/sec entries = 921893 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.59 avg-zval = 34.92 usrkeys = 921893 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.63 avg-zval = 34.92 TagRS{ kinds = RS_Key0_TagN bytes = 144160 } TagArray{ size = 664278 bytes = 2491056 } seq expand size = 4649946 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4445 value = 3.3673 dict = 1.00 all = 3.5613 } Zip/UnZip{ index = 0.0957 value = 0.2970 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 9.638372 GB avg = 0.163 KB (by entry num) total key len = 1.627559 GB avg = 0.028 KB total ukey len = 1.131263 GB avg = 0.016 KB total ukey num = 0.072219661 Billion total entry num = 0.059027390 Billion write speed all = 0.587849517 MB/sec (with seq num) write speed all = 0.563209408 MB/sec (without seq num) 2024/11/23-14:14:36.912608 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262587: 921893 keys, 37687640 bytes, temperature: kWarm 2024/11/23-14:14:36.912678 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342476912641, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262587, "file_size": 37687640, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012412200, "table_properties": {"data_size": 32193808, "index_size": 1765312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25813004, "raw_average_key_size": 28, "raw_value_size": 108404834, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921893, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342473, "oldest_key_time": 0, "file_creation_time": 1732342473, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262587, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:36.950070 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262588.sst) /dev/shm/ToplingTemp/Topling-69172-IMJCZE 2024/11/23-14:14:37.944205 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262588.sst): first pass time = 0.99's, 135.014'MB/sec 2024/11/23-14:14:39.270866 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262588.sst): old prealloc_size = 147639500, real_size = 32916432 2024/11/23-14:14:39.271429 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262588.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930941 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.17 avg-zval = 33.21 2024/11/23-14:14:39.272206 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262588.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:40.129454 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262588.sst): second pass time = 1.33's, 81.567'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.261'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3257.797'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.12's, 298.249'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28240.826'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.567'MB/sec zip pipeline throughput = 81.567'MB/sec entries = 930941 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.17 avg-zval = 33.21 usrkeys = 930941 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.17 avg-zval = 33.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930941 bytes = 3374672 } seq expand size = 6516587 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5139 value = 3.4986 dict = 1.00 all = 3.6136 } Zip/UnZip{ index = 0.0951 value = 0.2858 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 9.753040 GB avg = 0.163 KB (by entry num) total key len = 1.653626 GB avg = 0.028 KB total ukey len = 1.149881 GB avg = 0.016 KB total ukey num = 0.073150602 Billion total entry num = 0.059958331 Billion write speed all = 0.595091642 MB/sec (with seq num) write speed all = 0.570067185 MB/sec (without seq num) 2024/11/23-14:14:40.175070 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262588: 930941 keys, 37142358 bytes, temperature: kWarm 2024/11/23-14:14:40.175138 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342480175103, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262588, "file_size": 37142358, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012409018, "table_properties": {"data_size": 30912640, "index_size": 1770880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26066348, "raw_average_key_size": 28, "raw_value_size": 108151426, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930941, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342476, "oldest_key_time": 0, "file_creation_time": 1732342476, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262588, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:40.218654 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262589.sst) /dev/shm/ToplingTemp/Topling-69172-Ye0QEF 2024/11/23-14:14:41.217179 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262589.sst): first pass time = 1.00's, 134.421'MB/sec 2024/11/23-14:14:42.521606 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262589.sst): old prealloc_size = 147639500, real_size = 32892064 2024/11/23-14:14:42.522231 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262589.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932047 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.00 avg-zval = 33.14 2024/11/23-14:14:42.522998 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262589.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:43.379638 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262589.sst): second pass time = 1.30's, 82.936'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.129'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2955.776'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 293.708'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35579.875'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.936'MB/sec zip pipeline throughput = 82.936'MB/sec entries = 932047 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.00 avg-zval = 33.14 usrkeys = 932047 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.00 avg-zval = 33.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932047 bytes = 3262176 } seq expand size = 6524329 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5119 value = 3.5007 dict = 1.00 all = 3.6270 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2757 } ---------------------------- total value len = 9.867685 GB avg = 0.162 KB (by entry num) total key len = 1.679723 GB avg = 0.028 KB total ukey len = 1.168522 GB avg = 0.016 KB total ukey num = 0.074082649 Billion total entry num = 0.060890378 Billion write speed all = 0.602332094 MB/sec (with seq num) write speed all = 0.576922943 MB/sec (without seq num) 2024/11/23-14:14:43.425290 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262589: 932047 keys, 37004806 bytes, temperature: kWarm 2024/11/23-14:14:43.425367 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342483425332, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262589, "file_size": 37004806, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012412225, "table_properties": {"data_size": 30885584, "index_size": 1773312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26097316, "raw_average_key_size": 28, "raw_value_size": 108120418, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 932047, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342480, "oldest_key_time": 0, "file_creation_time": 1732342480, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262589, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:43.462747 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262590.sst) /dev/shm/ToplingTemp/Topling-69172-6GPZRF 2024/11/23-14:14:44.472070 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262590.sst): first pass time = 1.01's, 132.983'MB/sec 2024/11/23-14:14:45.782210 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262590.sst): old prealloc_size = 147639500, real_size = 33086416 2024/11/23-14:14:45.782763 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262590.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930708 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.21 avg-zval = 33.40 2024/11/23-14:14:45.784158 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262590.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:46.649652 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262590.sst): second pass time = 1.31's, 82.606'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.148'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3332.402'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 295.829'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38698.952'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.606'MB/sec zip pipeline throughput = 82.606'MB/sec entries = 930708 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.21 avg-zval = 33.40 usrkeys = 930708 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.21 avg-zval = 33.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930708 bytes = 3490176 } seq expand size = 6514956 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5139 value = 3.4796 dict = 1.00 all = 3.5860 } Zip/UnZip{ index = 0.0951 value = 0.2874 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 9.982358 GB avg = 0.161 KB (by entry num) total key len = 1.705783 GB avg = 0.028 KB total ukey len = 1.187136 GB avg = 0.016 KB total ukey num = 0.075013357 Billion total entry num = 0.061821086 Billion write speed all = 0.609568977 MB/sec (with seq num) write speed all = 0.583775847 MB/sec (without seq num) 2024/11/23-14:14:46.693909 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262590: 930708 keys, 37427774 bytes, temperature: kWarm 2024/11/23-14:14:46.693995 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342486693942, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262590, "file_size": 37427774, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012406444, "table_properties": {"data_size": 31083136, "index_size": 1770432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059824, "raw_average_key_size": 28, "raw_value_size": 108157992, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930708, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342483, "oldest_key_time": 0, "file_creation_time": 1732342483, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262590, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:46.735794 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262591.sst) /dev/shm/ToplingTemp/Topling-69172-BxEgFE 2024/11/23-14:14:47.772439 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262591.sst): first pass time = 1.04's, 129.477'MB/sec 2024/11/23-14:14:49.123131 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262591.sst): old prealloc_size = 147639500, real_size = 34567888 2024/11/23-14:14:49.123731 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262591.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928651 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.53 avg-zval = 35.05 2024/11/23-14:14:49.124544 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262591.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:49.970455 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262591.sst): second pass time = 1.35's, 80.168'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.778'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3145.754'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 302.934'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34033.721'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.168'MB/sec zip pipeline throughput = 80.168'MB/sec entries = 928651 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.53 avg-zval = 35.05 usrkeys = 928651 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.53 avg-zval = 35.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928651 bytes = 3366368 } seq expand size = 6500557 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.3770 value = 3.3250 dict = 1.00 all = 3.4603 } Zip/UnZip{ index = 0.0964 value = 0.3007 dict = 1.00 all = 0.2890 } ---------------------------- total value len = 10.097074 GB avg = 0.161 KB (by entry num) total key len = 1.731785 GB avg = 0.028 KB total ukey len = 1.205709 GB avg = 0.016 KB total ukey num = 0.075942008 Billion total entry num = 0.062749737 Billion write speed all = 0.616801010 MB/sec (with seq num) write speed all = 0.590624960 MB/sec (without seq num) 2024/11/23-14:14:50.017380 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262591: 928651 keys, 38787844 bytes, temperature: kWarm 2024/11/23-14:14:50.017447 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342490017411, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262591, "file_size": 38787844, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012412263, "table_properties": {"data_size": 32545728, "index_size": 1789824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002228, "raw_average_key_size": 28, "raw_value_size": 108215638, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928651, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342486, "oldest_key_time": 0, "file_creation_time": 1732342486, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262591, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:50.055542 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262592.sst) /dev/shm/ToplingTemp/Topling-69172-MFcPhH 2024/11/23-14:14:50.888135 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262592.sst): first pass time = 0.83's, 161.213'MB/sec 2024/11/23-14:14:52.167712 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262592.sst): old prealloc_size = 147639500, real_size = 30652592 2024/11/23-14:14:52.168314 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262592.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927236 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0286 GB avg-val = 116.75 avg-zval = 30.89 2024/11/23-14:14:52.169103 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262592.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:14:52.983656 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262592.sst): second pass time = 1.28's, 84.650'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.449'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3074.268'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 274.941'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35292.289'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.650'MB/sec zip pipeline throughput = 84.650'MB/sec entries = 927236 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.75 avg-zval = 30.89 usrkeys = 927236 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.75 avg-zval = 30.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927236 bytes = 3129440 } seq expand size = 6490652 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4449 value = 3.7792 dict = 1.00 all = 3.8752 } Zip/UnZip{ index = 0.0957 value = 0.2646 dict = 1.00 all = 0.2580 } ---------------------------- total value len = 10.211820 GB avg = 0.160 KB (by entry num) total key len = 1.757748 GB avg = 0.028 KB total ukey len = 1.224254 GB avg = 0.016 KB total ukey num = 0.076869244 Billion total entry num = 0.063676973 Billion write speed all = 0.624040025 MB/sec (with seq num) write speed all = 0.597481351 MB/sec (without seq num) 2024/11/23-14:14:53.026391 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262592: 927236 keys, 34634736 bytes, temperature: kWarm 2024/11/23-14:14:53.026458 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342493026423, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262592, "file_size": 34634736, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3887475725, "largest_seqno": 4012412279, "table_properties": {"data_size": 28645088, "index_size": 1775488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962608, "raw_average_key_size": 28, "raw_value_size": 108255156, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927236, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342490, "oldest_key_time": 0, "file_creation_time": 1732342490, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262592, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:53.063220 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262593.sst) /dev/shm/ToplingTemp/Topling-69172-8syTSF 2024/11/23-14:14:53.368356 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262593.sst): first pass time = 0.31's, 247.928'MB/sec 2024/11/23-14:14:54.154887 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262593.sst): old prealloc_size = 147639500, real_size = 20185392 2024/11/23-14:14:54.155277 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262593.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 522176 prefix = 4 raw-key = 0.0084 GB zip-key = 0.0010 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0610 GB zip-val = 0.0190 GB avg-val = 116.86 avg-zval = 36.48 2024/11/23-14:14:54.155752 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262593.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.076 GB 2024/11/23-14:14:54.627099 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00/262593.sst): second pass time = 0.79's, 77.656'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.47's, 47.783'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2755.330'MB/sec (index lex order gen) rebuild zvType time = 0.40's, 0.327'MB/sec write SST data time = 0.07's, 312.812'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.611'MB, throughput = 19483.593'MB/sec zip my value time = 0.79's, unzip length = 0.061'GB zip my value throughput = 77.656'MB/sec zip pipeline throughput = 77.656'MB/sec entries = 522176 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.86 avg-zval = 36.48 usrkeys = 522176 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.86 avg-zval = 36.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 522176 bytes = 1892896 } seq expand size = 3655232 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0104 GB value = 0.0610 GB dict = 0.61 MB all = 0.0756 GB } __ZipSize{ index = 0.0010 GB value = 0.0190 GB dict = 0.61 MB all = 0.0226 GB } UnZip/Zip{ index = 10.3705 value = 3.2036 dict = 1.00 all = 3.3525 } Zip/UnZip{ index = 0.0964 value = 0.3121 dict = 1.00 all = 0.2983 } ---------------------------- total value len = 10.276496 GB avg = 0.160 KB (by entry num) total key len = 1.772369 GB avg = 0.028 KB total ukey len = 1.234698 GB avg = 0.016 KB total ukey num = 0.077391420 Billion total entry num = 0.064199149 Billion write speed all = 0.628120412 MB/sec (with seq num) write speed all = 0.601346241 MB/sec (without seq num) 2024/11/23-14:14:54.657947 139957458552384 [compaction_job.cc:1948] [default] [JOB 27430] Generated table #262593: 522176 keys, 22562553 bytes, temperature: kWarm 2024/11/23-14:14:54.658028 139957458552384 EVENT_LOG_v1 {"time_micros": 1732342494657994, "cf_name": "default", "job": 27430, "event": "table_file_creation", "file_number": 262593, "file_size": 22562553, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 4012407454, "table_properties": {"data_size": 19047648, "index_size": 1007040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 14620928, "raw_average_key_size": 28, "raw_value_size": 61021080, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 522176, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342493, "oldest_key_time": 0, "file_creation_time": 1732342493, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262593, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:14:54.677051 139957458552384 [compaction_job.cc:703] job-27430: subcompact[0], size: 0.461294 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27430/att-00 [262581,262582,262583,262584,262585,262586,262587,262588,262589,262590,262591,262592,262593] 2024/11/23-14:14:54.754383 139957458552384 (Original Log Time 2024/11/23-14:14:14.074729) EVENT_LOG_v1 {"time_micros": 1732342454074690, "job": 27430, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [262509], "files_L3": [262563, 262564, 262469, 262430, 262431, 262432, 262433, 262434, 262435, 262436, 262437, 262438, 262439, 262565], "score": 0.994434, "input_data_size": 477677921, "oldest_snapshot_seqno": 4012741837} 2024/11/23-14:14:54.754386 139957458552384 (Original Log Time 2024/11/23-14:14:54.753295) [compaction_job.cc:2022] [default] [JOB 27430] Compacted 1@2 + 14@3 files to L3 => 461294375 bytes 2024/11/23-14:14:54.754388 139957458552384 (Original Log Time 2024/11/23-14:14:54.754318) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 11.8 rd, 11.4 wr, level 3, files in(1, 14) out(13 +0 blob) MB in(21.7, 433.9 +0.0 blob) out(439.9 +0.0 blob), read-write-amplify(41.3) write-amplify(20.3) OK, records in: 11873826, records dropped: 32841 output_compression: Snappy 2024/11/23-14:14:54.754394 139957458552384 (Original Log Time 2024/11/23-14:14:54.754362) EVENT_LOG_v1 {"time_micros": 1732342494754341, "cf": "default", "job": 27430, "event": "compaction_finished", "compaction_time_micros": 40602328, "compaction_time_cpu_micros": 17309884, "output_level": 3, "num_output_files": 13, "total_output_size": 461294375, "num_input_records": 11873826, "num_output_records": 11840985, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 313659663, "file_range_sync_nanos": 0, "file_fsync_nanos": 9878, "file_prepare_write_nanos": 31706, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/23-14:14:54.881180 139957458552384 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732342453,"executesMs":40834,"compactionJobId":27430,"attempt":0,"compactionInputRawBytes":1690919281,"compactionInputZipBytes":477677921,"compactionOutputRawBytes":0,"compactionOutputZipBytes":461294375,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5d954ed1c245871722791b2b221d6059-0c446604efc0d038-00"}}, response = 2024/11/23-14:14:54.881248 139957458552384 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27430/att-00: olev 3, work 40.835 s, result 7.706 ms, install 1.114 ms, input{raw 1.575 GiB zip 455.549 MiB}