2024/11/24-04:33:26.013369 139957643191872 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00: fork to child time = 0.009476 sec 2024/11/24-04:33:26.014187 139957643191872 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00 2024/11/24-04:33:26.022496 139957643191872 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00 2024/11/24-04:33:26.023436 139957643191872 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-04:33:26.023707 139957643191872 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-04:33:26.072637 139957643191872 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41598/att-00/MANIFEST-206347 2024/11/24-04:33:26.099451 139957643191872 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41598/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 292106, last_sequence is 4441120764, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 291949 2024/11/24-04:33:26.099460 139957643191872 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-04:33:26.099623 139957643191872 [version_set.cc:5721] Creating manifest 292106 2024/11/24-04:33:26.130403 139957643191872 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-04:33:26.152319 139957643191872 [compaction_job.cc:2437] [default] [JOB 41598] Compacting 1@2 + 21@3 files to L3, score 0.88, subcompactions 1 : 1 2024/11/24-04:33:26.152340 139957643191872 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31177 Base level 2, inputs: [292097(27MB)], [292018(25MB) 292019(33MB) 292020(35MB) 292021(33MB) 292022(33MB) 292057(35MB) 292058(34MB) 292059(33MB) 292060(35MB) 292061(34MB) 292062(2623KB) 292026(33MB) 292027(35MB) 292028(35MB) 292029(33MB) 292030(34MB) 292031(36MB) 292032(31MB) 292033(31MB) 292034(33MB) 292035(31MB)] 2024/11/24-04:33:26.160202 139957643191872 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:33:26.160238 139957643191872 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:33:26.160258 139957643191872 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:33:26.160263 139957643191872 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:33:26.160275 139957643191872 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:33:26.646186 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292107.sst) /dev/shm/ToplingTemp/Topling-270786-ZuvfML 2024/11/24-04:33:27.735962 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292107.sst): first pass time = 1.09's, 93.958'MB/sec 2024/11/24-04:33:29.631388 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292107.sst): old prealloc_size = 147639500, real_size = 24165216 2024/11/24-04:33:29.631938 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292107.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 709380 prefix = 4 raw-key = 0.0114 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0825 GB zip-val = 0.0226 GB avg-val = 116.33 avg-zval = 31.90 2024/11/24-04:33:29.632732 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292107.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0018 GB, waited 0.000 sec, Key+Value = 0.102 GB 2024/11/24-04:33:30.355324 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292107.sst): second pass time = 1.89's, 43.562'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 37.592'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2574.056'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.294'MB/sec write SST data time = 0.12's, 227.236'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.829'MB, throughput = 20059.431'MB/sec zip my value time = 1.89's, unzip length = 0.083'GB zip my value throughput = 43.562'MB/sec zip pipeline throughput = 80.553'MB/sec entries = 709380 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.33 avg-zval = 31.90 usrkeys = 709380 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.33 avg-zval = 31.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 709380 bytes = 2394176 } seq expand size = 4965660 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0142 GB value = 0.0825 GB dict = 0.83 MB all = 0.1024 GB } __ZipSize{ index = 0.0014 GB value = 0.0226 GB dict = 0.83 MB all = 0.0272 GB } UnZip/Zip{ index = 10.4424 value = 3.6468 dict = 1.00 all = 3.7622 } Zip/UnZip{ index = 0.0958 value = 0.2742 dict = 1.00 all = 0.2658 } ---------------------------- total value len = 2251.413062 GB avg = 0.200 KB (by entry num) total key len = 306.902825 GB avg = 0.027 KB total ukey len = 237.368915 GB avg = 0.015 KB total ukey num = 15.739370543 Billion total entry num = 11.280684491 Billion write speed all = 36.186466331 MB/sec (with seq num) write speed all = 34.909976241 MB/sec (without seq num) 2024/11/24-04:33:30.393793 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292107: 709380 keys, 27214316 bytes, temperature: kWarm 2024/11/24-04:33:30.393890 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394010393830, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292107, "file_size": 27214316, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4372064647, "largest_seqno": 4441097198, "table_properties": {"data_size": 22629008, "index_size": 1358656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19862640, "raw_average_key_size": 28, "raw_value_size": 82523120, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 709380, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394006, "oldest_key_time": 0, "file_creation_time": 1732394006, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292107, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:33:30.434077 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292108.sst) /dev/shm/ToplingTemp/Topling-270786-TmzOpK 2024/11/24-04:33:31.644425 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292108.sst): first pass time = 1.21's, 110.895'MB/sec 2024/11/24-04:33:33.443428 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292108.sst): old prealloc_size = 147639500, real_size = 31398672 2024/11/24-04:33:33.444007 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292108.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-04:33:33.444969 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292108.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:33:34.373187 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292108.sst): second pass time = 1.80's, 60.327'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 38.157'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3159.356'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.21's, 167.387'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34517.083'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.327'MB/sec zip pipeline throughput = 115.359'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6886 dict = 1.00 all = 3.7834 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2251.527974 GB avg = 0.200 KB (by entry num) total key len = 306.928566 GB avg = 0.027 KB total ukey len = 237.387301 GB avg = 0.015 KB total ukey num = 15.740289843 Billion total entry num = 11.281603791 Billion write speed all = 36.186399311 MB/sec (with seq num) write speed all = 34.909877742 MB/sec (without seq num) 2024/11/24-04:33:34.423588 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292108: 919300 keys, 35475378 bytes, temperature: kWarm 2024/11/24-04:33:34.423686 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394014423633, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292108, "file_size": 35475378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4414357721, "table_properties": {"data_size": 29408512, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394010, "oldest_key_time": 0, "file_creation_time": 1732394010, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292108, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:33:34.509851 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292109.sst) /dev/shm/ToplingTemp/Topling-270786-TAkhBM 2024/11/24-04:33:35.198708 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292109.sst): first pass time = 0.69's, 194.858'MB/sec 2024/11/24-04:33:36.593532 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292109.sst): old prealloc_size = 147639500, real_size = 33382368 2024/11/24-04:33:36.594113 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292109.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927209 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.75 avg-zval = 33.85 2024/11/24-04:33:36.594887 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292109.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:33:37.459143 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292109.sst): second pass time = 1.39's, 77.655'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.303'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3167.021'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 293.065'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36554.650'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.655'MB/sec zip pipeline throughput = 77.655'MB/sec entries = 927209 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 33.85 usrkeys = 927209 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 33.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927209 bytes = 3245248 } seq expand size = 6490463 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5112 value = 3.4492 dict = 1.00 all = 3.5808 } Zip/UnZip{ index = 0.0951 value = 0.2899 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 2251.642721 GB avg = 0.200 KB (by entry num) total key len = 306.954527 GB avg = 0.027 KB total ukey len = 237.405845 GB avg = 0.015 KB total ukey num = 15.741217052 Billion total entry num = 11.282531000 Billion write speed all = 36.186809999 MB/sec (with seq num) write speed all = 34.910239235 MB/sec (without seq num) 2024/11/24-04:33:37.506346 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292109: 927209 keys, 37482208 bytes, temperature: kWarm 2024/11/24-04:33:37.506415 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394017506380, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292109, "file_size": 37482208, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4441096482, "table_properties": {"data_size": 31386128, "index_size": 1764224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961852, "raw_average_key_size": 28, "raw_value_size": 108255902, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927209, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394014, "oldest_key_time": 0, "file_creation_time": 1732394014, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292109, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:33:37.545628 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292110.sst) /dev/shm/ToplingTemp/Topling-270786-jidA5J 2024/11/24-04:33:38.553777 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292110.sst): first pass time = 1.01's, 133.138'MB/sec 2024/11/24-04:33:39.900058 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292110.sst): old prealloc_size = 147639500, real_size = 31287312 2024/11/24-04:33:39.900638 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292110.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926836 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0293 GB avg-val = 116.81 avg-zval = 31.60 2024/11/24-04:33:39.901399 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292110.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:33:40.731687 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292110.sst): second pass time = 1.34's, 80.500'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.551'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.537'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 273.188'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34605.433'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.500'MB/sec zip pipeline throughput = 80.500'MB/sec entries = 926836 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 31.60 usrkeys = 926836 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 31.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926836 bytes = 3243936 } seq expand size = 6487852 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5127 value = 3.6961 dict = 1.00 all = 3.7930 } Zip/UnZip{ index = 0.0951 value = 0.2706 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2251.757475 GB avg = 0.200 KB (by entry num) total key len = 306.980479 GB avg = 0.027 KB total ukey len = 237.424382 GB avg = 0.015 KB total ukey num = 15.742143888 Billion total entry num = 11.283457836 Billion write speed all = 36.187125134 MB/sec (with seq num) write speed all = 34.910508589 MB/sec (without seq num) 2024/11/24-04:33:40.776342 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292110: 926836 keys, 35385534 bytes, temperature: kWarm 2024/11/24-04:33:40.776411 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394020776375, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292110, "file_size": 35385534, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4441096171, "table_properties": {"data_size": 29292160, "index_size": 1763264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951408, "raw_average_key_size": 28, "raw_value_size": 108266336, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926836, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394017, "oldest_key_time": 0, "file_creation_time": 1732394017, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292110, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:33:40.814326 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292111.sst) /dev/shm/ToplingTemp/Topling-270786-YGoYBL 2024/11/24-04:33:41.815288 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292111.sst): first pass time = 1.00's, 134.094'MB/sec 2024/11/24-04:33:43.148754 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292111.sst): old prealloc_size = 147639500, real_size = 31479280 2024/11/24-04:33:43.149342 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292111.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.08 2024/11/24-04:33:43.150092 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292111.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:33:43.965344 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292111.sst): second pass time = 1.33's, 81.396'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.544'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3114.818'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 278.607'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34251.215'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.396'MB/sec zip pipeline throughput = 81.396'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 32.08 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4424 value = 3.6786 dict = 1.00 all = 3.7748 } Zip/UnZip{ index = 0.0958 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2251.872388 GB avg = 0.200 KB (by entry num) total key len = 307.006219 GB avg = 0.027 KB total ukey len = 237.442768 GB avg = 0.015 KB total ukey num = 15.743063188 Billion total entry num = 11.284377136 Billion write speed all = 36.187459394 MB/sec (with seq num) write speed all = 34.910797224 MB/sec (without seq num) 2024/11/24-04:33:44.010181 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292111: 919300 keys, 35556018 bytes, temperature: kWarm 2024/11/24-04:33:44.010257 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394024010220, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292111, "file_size": 35556018, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4268575685, "largest_seqno": 4421680472, "table_properties": {"data_size": 29488544, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394020, "oldest_key_time": 0, "file_creation_time": 1732394020, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292111, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:33:44.047200 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292112.sst) /dev/shm/ToplingTemp/Topling-270786-J3ua1K 2024/11/24-04:33:44.888933 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292112.sst): first pass time = 0.84's, 159.462'MB/sec 2024/11/24-04:33:46.269439 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292112.sst): old prealloc_size = 147639500, real_size = 33481856 2024/11/24-04:33:46.270030 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292112.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926989 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.79 avg-zval = 33.97 2024/11/24-04:33:46.270785 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292112.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:33:47.136412 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292112.sst): second pass time = 1.38's, 78.506'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.348'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3114.687'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 291.506'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34190.434'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.506'MB/sec zip pipeline throughput = 78.506'MB/sec entries = 926989 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 33.97 usrkeys = 926989 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 33.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926989 bytes = 3244480 } seq expand size = 6488923 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5129 value = 3.4384 dict = 1.00 all = 3.5715 } Zip/UnZip{ index = 0.0951 value = 0.2908 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 2251.987139 GB avg = 0.200 KB (by entry num) total key len = 307.032175 GB avg = 0.027 KB total ukey len = 237.461308 GB avg = 0.015 KB total ukey num = 15.743990177 Billion total entry num = 11.285304125 Billion write speed all = 36.187826413 MB/sec (with seq num) write speed all = 34.911116622 MB/sec (without seq num) 2024/11/24-04:33:47.184068 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292112: 926989 keys, 37580542 bytes, temperature: kWarm 2024/11/24-04:33:47.184135 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394027184100, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292112, "file_size": 37580542, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4441096817, "table_properties": {"data_size": 31486384, "index_size": 1763520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955692, "raw_average_key_size": 28, "raw_value_size": 108262054, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926989, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394024, "oldest_key_time": 0, "file_creation_time": 1732394024, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292112, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:33:47.221788 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292113.sst) /dev/shm/ToplingTemp/Topling-270786-RfxTkM 2024/11/24-04:33:48.238014 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292113.sst): first pass time = 1.02's, 132.080'MB/sec 2024/11/24-04:33:49.574603 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292113.sst): old prealloc_size = 147639500, real_size = 31728800 2024/11/24-04:33:49.575192 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292113.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927168 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 116.76 avg-zval = 32.06 2024/11/24-04:33:49.575942 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292113.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:33:50.400256 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292113.sst): second pass time = 1.34's, 81.081'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.396'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3076.540'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 282.268'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33174.542'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.081'MB/sec zip pipeline throughput = 81.081'MB/sec entries = 927168 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.76 avg-zval = 32.06 usrkeys = 927168 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.76 avg-zval = 32.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927168 bytes = 3245104 } seq expand size = 6490176 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.4433 value = 3.6424 dict = 1.00 all = 3.7461 } Zip/UnZip{ index = 0.0958 value = 0.2745 dict = 1.00 all = 0.2669 } ---------------------------- total value len = 2252.101886 GB avg = 0.200 KB (by entry num) total key len = 307.058136 GB avg = 0.027 KB total ukey len = 237.479851 GB avg = 0.015 KB total ukey num = 15.744917345 Billion total entry num = 11.286231293 Billion write speed all = 36.188145944 MB/sec (with seq num) write speed all = 34.911390191 MB/sec (without seq num) 2024/11/24-04:33:50.445185 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292113: 927168 keys, 35828576 bytes, temperature: kWarm 2024/11/24-04:33:50.445263 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394030445226, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292113, "file_size": 35828576, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4441096757, "table_properties": {"data_size": 29721232, "index_size": 1775616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960704, "raw_average_key_size": 28, "raw_value_size": 108257104, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927168, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394027, "oldest_key_time": 0, "file_creation_time": 1732394027, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292113, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:33:50.481347 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292114.sst) /dev/shm/ToplingTemp/Topling-270786-QdPdeL 2024/11/24-04:33:51.447360 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292114.sst): first pass time = 0.97's, 138.946'MB/sec 2024/11/24-04:33:52.767518 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292114.sst): old prealloc_size = 147639500, real_size = 31516912 2024/11/24-04:33:52.768065 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292114.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.12 2024/11/24-04:33:52.768814 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292114.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:33:53.587702 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292114.sst): second pass time = 1.32's, 82.249'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.399'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3363.131'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.331'MB/sec write SST data time = 0.13's, 282.621'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34032.056'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.249'MB/sec zip pipeline throughput = 82.249'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.12 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4458 value = 3.6739 dict = 1.00 all = 3.7708 } Zip/UnZip{ index = 0.0957 value = 0.2722 dict = 1.00 all = 0.2652 } ---------------------------- total value len = 2252.216798 GB avg = 0.200 KB (by entry num) total key len = 307.083876 GB avg = 0.027 KB total ukey len = 237.498237 GB avg = 0.015 KB total ukey num = 15.745836645 Billion total entry num = 11.287150593 Billion write speed all = 36.188503757 MB/sec (with seq num) write speed all = 34.911701556 MB/sec (without seq num) 2024/11/24-04:33:53.632092 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292114: 919300 keys, 35593650 bytes, temperature: kWarm 2024/11/24-04:33:53.632163 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394033632126, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292114, "file_size": 35593650, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4275788504, "largest_seqno": 4428019326, "table_properties": {"data_size": 29526752, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394030, "oldest_key_time": 0, "file_creation_time": 1732394030, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292114, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:33:53.671132 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292115.sst) /dev/shm/ToplingTemp/Topling-270786-uocKHL 2024/11/24-04:33:54.706066 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292115.sst): first pass time = 1.03's, 129.691'MB/sec 2024/11/24-04:33:56.048725 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292115.sst): old prealloc_size = 147639500, real_size = 33347232 2024/11/24-04:33:56.049304 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292115.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927044 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.78 avg-zval = 33.82 2024/11/24-04:33:56.050103 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292115.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:33:56.903879 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292115.sst): second pass time = 1.34's, 80.704'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.790'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3168.066'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 294.152'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34655.866'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.704'MB/sec zip pipeline throughput = 80.704'MB/sec entries = 927044 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 33.82 usrkeys = 927044 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.78 avg-zval = 33.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927044 bytes = 3244672 } seq expand size = 6489308 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5117 value = 3.4531 dict = 1.00 all = 3.5843 } Zip/UnZip{ index = 0.0951 value = 0.2896 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 2252.331548 GB avg = 0.200 KB (by entry num) total key len = 307.109833 GB avg = 0.027 KB total ukey len = 237.516778 GB avg = 0.015 KB total ukey num = 15.746763689 Billion total entry num = 11.288077637 Billion write speed all = 36.188796435 MB/sec (with seq num) write speed all = 34.911949240 MB/sec (without seq num) 2024/11/24-04:33:56.949660 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292115: 927044 keys, 37446226 bytes, temperature: kWarm 2024/11/24-04:33:56.949728 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394036949696, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292115, "file_size": 37446226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4441096580, "table_properties": {"data_size": 31351440, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957232, "raw_average_key_size": 28, "raw_value_size": 108260600, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927044, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394033, "oldest_key_time": 0, "file_creation_time": 1732394033, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292115, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:33:56.987363 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292116.sst) /dev/shm/ToplingTemp/Topling-270786-5jNWTK 2024/11/24-04:33:57.925233 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292116.sst): first pass time = 0.94's, 143.115'MB/sec 2024/11/24-04:33:59.267821 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292116.sst): old prealloc_size = 147639500, real_size = 31953552 2024/11/24-04:33:59.268404 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292116.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927421 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0300 GB avg-val = 116.72 avg-zval = 32.30 2024/11/24-04:33:59.269187 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292116.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:00.105117 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292116.sst): second pass time = 1.34's, 80.673'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.061'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3160.173'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 283.214'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30871.292'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.673'MB/sec zip pipeline throughput = 80.673'MB/sec entries = 927421 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.72 avg-zval = 32.30 usrkeys = 927421 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.72 avg-zval = 32.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927421 bytes = 3245984 } seq expand size = 6491947 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5125 value = 3.6135 dict = 1.00 all = 3.7227 } Zip/UnZip{ index = 0.0951 value = 0.2767 dict = 1.00 all = 0.2686 } ---------------------------- total value len = 2252.446290 GB avg = 0.200 KB (by entry num) total key len = 307.135801 GB avg = 0.027 KB total ukey len = 237.535326 GB avg = 0.015 KB total ukey num = 15.747691110 Billion total entry num = 11.289005058 Billion write speed all = 36.189147937 MB/sec (with seq num) write speed all = 34.912253633 MB/sec (without seq num) 2024/11/24-04:34:00.150161 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292116: 927421 keys, 36053830 bytes, temperature: kWarm 2024/11/24-04:34:00.150255 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394040150197, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292116, "file_size": 36053830, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4441096220, "table_properties": {"data_size": 29957120, "index_size": 1764416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25967788, "raw_average_key_size": 28, "raw_value_size": 108250038, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927421, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394036, "oldest_key_time": 0, "file_creation_time": 1732394036, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292116, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:00.188306 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292117.sst) /dev/shm/ToplingTemp/Topling-270786-FtABPM 2024/11/24-04:34:01.223528 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292117.sst): first pass time = 1.04's, 129.656'MB/sec 2024/11/24-04:34:02.551802 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292117.sst): old prealloc_size = 147639500, real_size = 31507008 2024/11/24-04:34:02.552384 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292117.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.11 2024/11/24-04:34:02.553172 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292117.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:03.381346 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292117.sst): second pass time = 1.33's, 81.736'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.897'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3137.048'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 278.037'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35467.116'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.736'MB/sec zip pipeline throughput = 81.736'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.11 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4466 value = 3.6751 dict = 1.00 all = 3.7719 } Zip/UnZip{ index = 0.0957 value = 0.2721 dict = 1.00 all = 0.2651 } ---------------------------- total value len = 2252.561203 GB avg = 0.200 KB (by entry num) total key len = 307.161541 GB avg = 0.027 KB total ukey len = 237.553712 GB avg = 0.015 KB total ukey num = 15.748610410 Billion total entry num = 11.289924358 Billion write speed all = 36.189460232 MB/sec (with seq num) write speed all = 34.912521096 MB/sec (without seq num) 2024/11/24-04:34:03.425867 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292117: 919300 keys, 35583730 bytes, temperature: kWarm 2024/11/24-04:34:03.425936 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394043425899, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292117, "file_size": 35583730, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4280702968, "largest_seqno": 4432743175, "table_properties": {"data_size": 29516976, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394040, "oldest_key_time": 0, "file_creation_time": 1732394040, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292117, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:03.463364 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292118.sst) /dev/shm/ToplingTemp/Topling-270786-9IBAoJ 2024/11/24-04:34:04.484665 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292118.sst): first pass time = 1.02's, 131.423'MB/sec 2024/11/24-04:34:05.827375 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292118.sst): old prealloc_size = 147639500, real_size = 33277648 2024/11/24-04:34:05.827922 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292118.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927072 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.78 avg-zval = 33.73 2024/11/24-04:34:05.829614 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292118.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:06.686079 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292118.sst): second pass time = 1.34's, 80.676'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.527'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3387.270'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 291.927'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32467.741'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.676'MB/sec zip pipeline throughput = 80.676'MB/sec entries = 927072 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.78 avg-zval = 33.73 usrkeys = 927072 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.78 avg-zval = 33.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927072 bytes = 3244768 } seq expand size = 6489504 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4434 value = 3.4621 dict = 1.00 all = 3.5910 } Zip/UnZip{ index = 0.0958 value = 0.2888 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 2252.675952 GB avg = 0.200 KB (by entry num) total key len = 307.187499 GB avg = 0.027 KB total ukey len = 237.572254 GB avg = 0.015 KB total ukey num = 15.749537482 Billion total entry num = 11.290851430 Billion write speed all = 36.189758683 MB/sec (with seq num) write speed all = 34.912774355 MB/sec (without seq num) 2024/11/24-04:34:06.732745 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292118: 927072 keys, 37376470 bytes, temperature: kWarm 2024/11/24-04:34:06.732816 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394046732779, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292118, "file_size": 37376470, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4441096236, "table_properties": {"data_size": 31270272, "index_size": 1775424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958016, "raw_average_key_size": 28, "raw_value_size": 108259756, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927072, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394043, "oldest_key_time": 0, "file_creation_time": 1732394043, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292118, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:06.774345 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292119.sst) /dev/shm/ToplingTemp/Topling-270786-p4otgL 2024/11/24-04:34:07.743025 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292119.sst): first pass time = 0.97's, 138.563'MB/sec 2024/11/24-04:34:09.089423 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292119.sst): old prealloc_size = 147639500, real_size = 32539616 2024/11/24-04:34:09.089954 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292119.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927160 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 116.76 avg-zval = 32.94 2024/11/24-04:34:09.091828 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292119.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:09.936748 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292119.sst): second pass time = 1.35's, 80.483'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.242'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3466.974'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 288.022'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32872.988'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.483'MB/sec zip pipeline throughput = 80.483'MB/sec entries = 927160 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 32.94 usrkeys = 927160 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 32.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927160 bytes = 3245072 } seq expand size = 6490120 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5137 value = 3.5443 dict = 1.00 all = 3.6632 } Zip/UnZip{ index = 0.0951 value = 0.2821 dict = 1.00 all = 0.2730 } ---------------------------- total value len = 2252.790699 GB avg = 0.200 KB (by entry num) total key len = 307.213460 GB avg = 0.027 KB total ukey len = 237.590797 GB avg = 0.015 KB total ukey num = 15.750464642 Billion total entry num = 11.291778590 Billion write speed all = 36.190084770 MB/sec (with seq num) write speed all = 34.913054269 MB/sec (without seq num) 2024/11/24-04:34:09.981196 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292119: 927160 keys, 36639304 bytes, temperature: kWarm 2024/11/24-04:34:09.981272 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394049981239, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292119, "file_size": 36639304, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4441096255, "table_properties": {"data_size": 30543952, "index_size": 1763712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960480, "raw_average_key_size": 28, "raw_value_size": 108257260, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927160, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394046, "oldest_key_time": 0, "file_creation_time": 1732394046, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292119, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:10.029320 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292120.sst) /dev/shm/ToplingTemp/Topling-270786-lhhrCK 2024/11/24-04:34:11.024330 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292120.sst): first pass time = 0.99's, 134.896'MB/sec 2024/11/24-04:34:12.337968 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292120.sst): old prealloc_size = 147639500, real_size = 31297344 2024/11/24-04:34:12.338526 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292120.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920401 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.83 avg-zval = 31.84 2024/11/24-04:34:12.339290 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292120.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:13.147699 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292120.sst): second pass time = 1.31's, 82.604'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.691'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3419.537'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 281.527'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31605.071'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.604'MB/sec zip pipeline throughput = 82.604'MB/sec entries = 920401 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.83 avg-zval = 31.84 usrkeys = 920401 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.83 avg-zval = 31.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920401 bytes = 3221424 } seq expand size = 6442807 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4450 value = 3.7007 dict = 1.00 all = 3.7939 } Zip/UnZip{ index = 0.0957 value = 0.2702 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2252.905589 GB avg = 0.200 KB (by entry num) total key len = 307.239231 GB avg = 0.027 KB total ukey len = 237.609205 GB avg = 0.015 KB total ukey num = 15.751385043 Billion total entry num = 11.292698991 Billion write speed all = 36.190430486 MB/sec (with seq num) write speed all = 34.913353863 MB/sec (without seq num) 2024/11/24-04:34:13.190367 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292120: 920401 keys, 35376994 bytes, temperature: kWarm 2024/11/24-04:34:13.190439 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394053190404, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292120, "file_size": 35376994, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4295719884, "largest_seqno": 4441096265, "table_properties": {"data_size": 29304688, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25771228, "raw_average_key_size": 28, "raw_value_size": 108446610, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920401, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394050, "oldest_key_time": 0, "file_creation_time": 1732394050, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292120, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:13.232393 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292121.sst) /dev/shm/ToplingTemp/Topling-270786-A6jsEJ 2024/11/24-04:34:14.236179 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292121.sst): first pass time = 1.00's, 133.716'MB/sec 2024/11/24-04:34:15.583047 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292121.sst): old prealloc_size = 147639500, real_size = 33115504 2024/11/24-04:34:15.583594 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292121.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925724 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.99 avg-zval = 33.61 2024/11/24-04:34:15.584395 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292121.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:16.436390 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292121.sst): second pass time = 1.35's, 80.454'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.606'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3393.640'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 297.176'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35502.416'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.454'MB/sec zip pipeline throughput = 80.454'MB/sec entries = 925724 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.99 avg-zval = 33.61 usrkeys = 925724 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.99 avg-zval = 33.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925724 bytes = 3240048 } seq expand size = 6480068 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4463 value = 3.4809 dict = 1.00 all = 3.6070 } Zip/UnZip{ index = 0.0957 value = 0.2873 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 2253.020366 GB avg = 0.199 KB (by entry num) total key len = 307.265151 GB avg = 0.027 KB total ukey len = 237.627719 GB avg = 0.015 KB total ukey num = 15.752310767 Billion total entry num = 11.293624715 Billion write speed all = 36.190736924 MB/sec (with seq num) write speed all = 34.913614984 MB/sec (without seq num) 2024/11/24-04:34:16.482308 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292121: 925724 keys, 37210026 bytes, temperature: kWarm 2024/11/24-04:34:16.482379 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394056482343, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292121, "file_size": 37210026, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4441096924, "table_properties": {"data_size": 31111520, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920272, "raw_average_key_size": 28, "raw_value_size": 108297480, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925724, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394053, "oldest_key_time": 0, "file_creation_time": 1732394053, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292121, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:16.521311 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292122.sst) /dev/shm/ToplingTemp/Topling-270786-nG78tN 2024/11/24-04:34:17.416594 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292122.sst): first pass time = 0.90's, 149.923'MB/sec 2024/11/24-04:34:18.763290 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292122.sst): old prealloc_size = 147639500, real_size = 33335600 2024/11/24-04:34:18.763864 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292122.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927258 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.75 avg-zval = 33.79 2024/11/24-04:34:18.765956 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292122.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:19.613163 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292122.sst): second pass time = 1.35's, 80.434'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.054'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3204.944'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 290.450'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32229.959'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.434'MB/sec zip pipeline throughput = 80.434'MB/sec entries = 927258 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.75 avg-zval = 33.79 usrkeys = 927258 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.75 avg-zval = 33.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927258 bytes = 3245424 } seq expand size = 6490806 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4447 value = 3.4555 dict = 1.00 all = 3.5850 } Zip/UnZip{ index = 0.0957 value = 0.2894 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 2253.135112 GB avg = 0.199 KB (by entry num) total key len = 307.291115 GB avg = 0.027 KB total ukey len = 237.646265 GB avg = 0.015 KB total ukey num = 15.753238025 Billion total entry num = 11.294551973 Billion write speed all = 36.191100737 MB/sec (with seq num) write speed all = 34.913931291 MB/sec (without seq num) 2024/11/24-04:34:19.659306 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292122: 927258 keys, 37439180 bytes, temperature: kWarm 2024/11/24-04:34:19.659379 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394059659343, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292122, "file_size": 37439180, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4441096882, "table_properties": {"data_size": 31328032, "index_size": 1775552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963224, "raw_average_key_size": 28, "raw_value_size": 108254520, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927258, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394056, "oldest_key_time": 0, "file_creation_time": 1732394056, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292122, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:19.695929 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292123.sst) /dev/shm/ToplingTemp/Topling-270786-rBMWVK 2024/11/24-04:34:20.669115 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292123.sst): first pass time = 0.97's, 137.920'MB/sec 2024/11/24-04:34:21.967695 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292123.sst): old prealloc_size = 147639500, real_size = 29697776 2024/11/24-04:34:21.968332 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292123.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934420 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0277 GB avg-val = 115.64 avg-zval = 29.62 2024/11/24-04:34:21.970405 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292123.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:22.800569 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292123.sst): second pass time = 1.30's, 83.287'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 40.604'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2902.503'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 262.916'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27616.501'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.287'MB/sec zip pipeline throughput = 83.287'MB/sec entries = 934420 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.64 avg-zval = 29.62 usrkeys = 934420 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.64 avg-zval = 29.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934420 bytes = 3270480 } seq expand size = 6540940 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0277 GB dict = 1.08 MB all = 0.0338 GB } UnZip/Zip{ index = 10.4445 value = 3.9044 dict = 1.00 all = 3.9689 } Zip/UnZip{ index = 0.0957 value = 0.2561 dict = 1.00 all = 0.2520 } ---------------------------- total value len = 2253.249707 GB avg = 0.199 KB (by entry num) total key len = 307.317278 GB avg = 0.027 KB total ukey len = 237.664953 GB avg = 0.015 KB total ukey num = 15.754172445 Billion total entry num = 11.295486393 Billion write speed all = 36.191459787 MB/sec (with seq num) write speed all = 34.914242221 MB/sec (without seq num) 2024/11/24-04:34:22.844558 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292123: 934420 keys, 33817092 bytes, temperature: kWarm 2024/11/24-04:34:22.844628 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394062844594, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292123, "file_size": 33817092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4441095412, "table_properties": {"data_size": 27674656, "index_size": 1789312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26163760, "raw_average_key_size": 28, "raw_value_size": 108054052, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934420, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394059, "oldest_key_time": 0, "file_creation_time": 1732394059, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292123, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:22.884109 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292124.sst) /dev/shm/ToplingTemp/Topling-270786-dpbh4L 2024/11/24-04:34:23.801936 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292124.sst): first pass time = 0.92's, 146.241'MB/sec 2024/11/24-04:34:25.060563 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292124.sst): old prealloc_size = 147639500, real_size = 27822304 2024/11/24-04:34:25.061125 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292124.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1078 GB zip-val = 0.0258 GB avg-val = 114.00 avg-zval = 27.26 2024/11/24-04:34:25.061999 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292124.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:25.898188 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292124.sst): second pass time = 1.26's, 85.701'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 38.029'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3389.841'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.332'MB/sec write SST data time = 0.13's, 253.471'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 32546.739'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.701'MB/sec zip pipeline throughput = 85.701'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.00 avg-zval = 27.26 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.00 avg-zval = 27.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0258 GB dict = 1.07 MB all = 0.0319 GB } UnZip/Zip{ index = 10.3782 value = 4.1822 dict = 1.00 all = 4.2136 } Zip/UnZip{ index = 0.0964 value = 0.2391 dict = 1.00 all = 0.2373 } ---------------------------- total value len = 2253.364075 GB avg = 0.199 KB (by entry num) total key len = 307.343744 GB avg = 0.027 KB total ukey len = 237.683857 GB avg = 0.015 KB total ukey num = 15.755117641 Billion total entry num = 11.296431589 Billion write speed all = 36.191865796 MB/sec (with seq num) write speed all = 34.914597275 MB/sec (without seq num) 2024/11/24-04:34:25.937173 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292124: 945196 keys, 31853488 bytes, temperature: kWarm 2024/11/24-04:34:25.937251 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394065937214, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292124, "file_size": 31853488, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4323493001, "largest_seqno": 4425591753, "table_properties": {"data_size": 25764304, "index_size": 1821504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394062, "oldest_key_time": 0, "file_creation_time": 1732394062, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292124, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:25.973110 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292125.sst) /dev/shm/ToplingTemp/Topling-270786-ifEzmL 2024/11/24-04:34:26.968007 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292125.sst): first pass time = 0.99's, 134.911'MB/sec 2024/11/24-04:34:28.306341 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292125.sst): old prealloc_size = 147639500, real_size = 32636672 2024/11/24-04:34:28.306910 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292125.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923175 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0306 GB avg-val = 117.39 avg-zval = 33.20 2024/11/24-04:34:28.309015 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292125.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:29.148132 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292125.sst): second pass time = 1.34's, 81.018'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.629'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.606'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 283.726'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35006.047'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.018'MB/sec zip pipeline throughput = 81.018'MB/sec entries = 923175 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 33.20 usrkeys = 923175 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 33.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923175 bytes = 3231120 } seq expand size = 6462225 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5116 value = 3.5358 dict = 1.00 all = 3.6546 } Zip/UnZip{ index = 0.0951 value = 0.2828 dict = 1.00 all = 0.2736 } ---------------------------- total value len = 2253.478906 GB avg = 0.199 KB (by entry num) total key len = 307.369593 GB avg = 0.027 KB total ukey len = 237.702320 GB avg = 0.015 KB total ukey num = 15.756040816 Billion total entry num = 11.297354764 Billion write speed all = 36.192191684 MB/sec (with seq num) write speed all = 34.914877452 MB/sec (without seq num) 2024/11/24-04:34:29.194065 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292125: 923175 keys, 36725324 bytes, temperature: kWarm 2024/11/24-04:34:29.194136 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394069194100, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292125, "file_size": 36725324, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4441095707, "table_properties": {"data_size": 30649200, "index_size": 1756480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25848900, "raw_average_key_size": 28, "raw_value_size": 108368946, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923175, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394065, "oldest_key_time": 0, "file_creation_time": 1732394065, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292125, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:29.230953 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292126.sst) /dev/shm/ToplingTemp/Topling-270786-EJKLwJ 2024/11/24-04:34:30.193054 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292126.sst): first pass time = 0.96's, 139.513'MB/sec 2024/11/24-04:34:31.496538 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292126.sst): old prealloc_size = 147639500, real_size = 29485984 2024/11/24-04:34:31.497100 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292126.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935768 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0275 GB avg-val = 115.43 avg-zval = 29.34 2024/11/24-04:34:31.499107 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292126.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:34:32.320272 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292126.sst): second pass time = 1.30's, 82.919'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 40.660'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3313.738'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 258.085'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33338.242'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.919'MB/sec zip pipeline throughput = 82.919'MB/sec entries = 935768 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.43 avg-zval = 29.34 usrkeys = 935768 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.43 avg-zval = 29.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935768 bytes = 3158224 } seq expand size = 6550376 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0275 GB dict = 1.08 MB all = 0.0335 GB } UnZip/Zip{ index = 10.4438 value = 3.9336 dict = 1.00 all = 4.0075 } Zip/UnZip{ index = 0.0958 value = 0.2542 dict = 1.00 all = 0.2495 } ---------------------------- total value len = 2253.593473 GB avg = 0.199 KB (by entry num) total key len = 307.395794 GB avg = 0.027 KB total ukey len = 237.721036 GB avg = 0.015 KB total ukey num = 15.756976584 Billion total entry num = 11.298290532 Billion write speed all = 36.192558577 MB/sec (with seq num) write speed all = 34.915195811 MB/sec (without seq num) 2024/11/24-04:34:32.362242 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292126: 935768 keys, 33491816 bytes, temperature: kWarm 2024/11/24-04:34:32.362311 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394072362276, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292126, "file_size": 33491816, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4325175635, "largest_seqno": 4441027914, "table_properties": {"data_size": 27459856, "index_size": 1792000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26201504, "raw_average_key_size": 28, "raw_value_size": 108016248, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935768, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394069, "oldest_key_time": 0, "file_creation_time": 1732394069, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292126, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:32.406323 139957643191872 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292127.sst) /dev/shm/ToplingTemp/Topling-270786-XDFlfN 2024/11/24-04:34:32.509529 139957643191872 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292127.sst): first pass time = 0.10's, 244.992'MB/sec 2024/11/24-04:34:32.745631 139957643191872 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292127.sst): old prealloc_size = 147639500, real_size = 6709744 2024/11/24-04:34:32.745779 139957643191872 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292127.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 177994 prefix = 4 raw-key = 0.0028 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0203 GB zip-val = 0.0063 GB avg-val = 114.00 avg-zval = 35.52 2024/11/24-04:34:32.745957 139957643191872 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292127.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.025 GB 2024/11/24-04:34:32.905926 139957643191872 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00/292127.sst): second pass time = 0.24's, 86.194'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 46.399'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2748.488'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.328'MB/sec write SST data time = 0.02's, 301.216'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.213'MB, throughput = 7201.784'MB/sec zip my value time = 0.24's, unzip length = 0.020'GB zip my value throughput = 86.194'MB/sec zip pipeline throughput = 86.194'MB/sec entries = 177994 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.00 avg-zval = 35.52 usrkeys = 177994 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.00 avg-zval = 35.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 177994 bytes = 556240 } seq expand size = 1245958 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0036 GB value = 0.0203 GB dict = 0.21 MB all = 0.0253 GB } __ZipSize{ index = 0.0003 GB value = 0.0063 GB dict = 0.21 MB all = 0.0074 GB } UnZip/Zip{ index = 10.4007 value = 3.2092 dict = 1.00 all = 3.3988 } Zip/UnZip{ index = 0.0961 value = 0.3116 dict = 1.00 all = 0.2942 } ---------------------------- total value len = 2253.615010 GB avg = 0.199 KB (by entry num) total key len = 307.400778 GB avg = 0.027 KB total ukey len = 237.724596 GB avg = 0.015 KB total ukey num = 15.757154578 Billion total entry num = 11.298468526 Billion write speed all = 36.192633824 MB/sec (with seq num) write speed all = 34.915261507 MB/sec (without seq num) 2024/11/24-04:34:32.923148 139957643191872 [compaction_job.cc:1948] [default] [JOB 41598] Generated table #292127: 177994 keys, 7436443 bytes, temperature: kWarm 2024/11/24-04:34:32.923213 139957643191872 EVENT_LOG_v1 {"time_micros": 1732394072923175, "cf_name": "default", "job": 41598, "event": "table_file_creation", "file_number": 292127, "file_size": 7436443, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4390036354, "largest_seqno": 4409353429, "table_properties": {"data_size": 6322784, "index_size": 342272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4983832, "raw_average_key_size": 28, "raw_value_size": 20291316, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 177994, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394072, "oldest_key_time": 0, "file_creation_time": 1732394072, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292127, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:34:32.936082 139957643191872 [compaction_job.cc:703] job-41598: subcompact[0], size: 0.716566 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41598/att-00 [292107,292108,292109,292110,292111,292112,292113,292114,292115,292116,292117,292118,292119,292120,292121,292122,292123,292124,292125,292126,292127] 2024/11/24-04:34:33.059827 139957643191872 (Original Log Time 2024/11/24-04:33:26.152408) EVENT_LOG_v1 {"time_micros": 1732394006152363, "job": 41598, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [292097], "files_L3": [292018, 292019, 292020, 292021, 292022, 292057, 292058, 292059, 292060, 292061, 292062, 292026, 292027, 292028, 292029, 292030, 292031, 292032, 292033, 292034, 292035], "score": 0.875857, "input_data_size": 737767096, "oldest_snapshot_seqno": 4441120686} 2024/11/24-04:34:33.059836 139957643191872 (Original Log Time 2024/11/24-04:34:33.057677) [compaction_job.cc:2022] [default] [JOB 41598] Compacted 1@2 + 21@3 files to L3 => 716566145 bytes 2024/11/24-04:34:33.059837 139957643191872 (Original Log Time 2024/11/24-04:34:33.059181) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 11.0 rd, 10.7 wr, level 3, files in(1, 21) out(21 +0 blob) MB in(27.1, 676.5 +0.0 blob) out(683.4 +0.0 blob), read-write-amplify(51.2) write-amplify(25.2) OK, records in: 18596325, records dropped: 102910 output_compression: Snappy 2024/11/24-04:34:33.059840 139957643191872 (Original Log Time 2024/11/24-04:34:33.059797) EVENT_LOG_v1 {"time_micros": 1732394073059211, "cf": "default", "job": 41598, "event": "compaction_finished", "compaction_time_micros": 66783681, "compaction_time_cpu_micros": 28434149, "output_level": 3, "num_output_files": 21, "total_output_size": 716566145, "num_input_records": 18596325, "num_output_records": 18493415, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 499700012, "file_range_sync_nanos": 0, "file_fsync_nanos": 14940, "file_prepare_write_nanos": 49359, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/24-04:34:33.192380 139957643191872 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732394005,"executesMs":67036,"compactionJobId":41598,"attempt":0,"compactionInputRawBytes":2692412126,"compactionInputZipBytes":737767096,"compactionOutputRawBytes":0,"compactionOutputZipBytes":716566145,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8612e8209d4257720e6eb8c49bad6b50-361b73ee4e11b698-00"}}, response = 2024/11/24-04:34:33.192453 139957643191872 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41598/att-00: olev 3, work 67.036 s, result 7.761 ms, install 2.178 ms, input{raw 2.508 GiB zip 703.590 MiB}