2024/11/24-01:59:32.821541 139957458552384 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00: fork to child time = 0.007706 sec 2024/11/24-01:59:32.822353 139957458552384 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00 2024/11/24-01:59:32.827920 139957458552384 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00 2024/11/24-01:59:32.828858 139957458552384 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-01:59:32.829263 139957458552384 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-01:59:32.884883 139957458552384 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39738/att-00/MANIFEST-206347 2024/11/24-01:59:32.910724 139957458552384 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39738/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 288070, last_sequence is 4384613952, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 287890 2024/11/24-01:59:32.910730 139957458552384 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-01:59:32.910968 139957458552384 [version_set.cc:5721] Creating manifest 288070 2024/11/24-01:59:32.945590 139957458552384 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-01:59:32.966525 139957458552384 [compaction_job.cc:2437] [default] [JOB 39738] Compacting 1@2 + 17@3 files to L3, score 0.77, subcompactions 1 : 1 2024/11/24-01:59:32.966539 139957458552384 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 29774 Base level 2, inputs: [288022(27MB)], [288067(34MB) 288068(953KB) 287979(34MB) 287980(35MB) 287981(35MB) 287982(35MB) 287983(35MB) 287984(35MB) 287985(35MB) 287986(4853KB) 287957(35MB) 287958(31MB) 287959(33MB) 287960(33MB) 287961(33MB) 287962(35MB) 288044(34MB)] 2024/11/24-01:59:32.973187 139957458552384 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:59:32.973217 139957458552384 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:59:32.973237 139957458552384 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:59:32.973242 139957458552384 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:59:32.973253 139957458552384 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:59:33.599023 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288071.sst) /dev/shm/ToplingTemp/Topling-243219-yjdjaf 2024/11/24-01:59:34.141501 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288071.sst): first pass time = 0.54's, 247.434'MB/sec 2024/11/24-01:59:35.594352 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288071.sst): old prealloc_size = 147639500, real_size = 32330256 2024/11/24-01:59:35.594883 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288071.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924553 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 117.17 avg-zval = 32.82 2024/11/24-01:59:35.595655 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288071.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:59:36.420587 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288071.sst): second pass time = 1.45's, 74.608'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.943'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3429.358'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.12's, 294.023'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33534.826'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.608'MB/sec zip pipeline throughput = 74.608'MB/sec entries = 924553 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.17 avg-zval = 32.82 usrkeys = 924553 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.17 avg-zval = 32.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924553 bytes = 3120384 } seq expand size = 6471871 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5113 value = 3.5706 dict = 1.00 all = 3.6968 } Zip/UnZip{ index = 0.0951 value = 0.2801 dict = 1.00 all = 0.2705 } ---------------------------- total value len = 8.963422 GB avg = 0.168 KB (by entry num) total key len = 1.466709 GB avg = 0.028 KB total ukey len = 1.016369 GB avg = 0.015 KB total ukey num = 0.066474988 Billion total entry num = 0.053282717 Billion write speed all = 0.169694384 MB/sec (with seq num) write speed all = 0.162759262 MB/sec (without seq num) 2024/11/24-01:59:36.467645 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288071: 924553 keys, 36306724 bytes, temperature: kWarm 2024/11/24-01:59:36.467740 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384776467680, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288071, "file_size": 36306724, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4384239835, "table_properties": {"data_size": 30339776, "index_size": 1759168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25887484, "raw_average_key_size": 28, "raw_value_size": 108330322, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924553, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384773, "oldest_key_time": 0, "file_creation_time": 1732384773, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288071, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:59:36.505647 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288072.sst) /dev/shm/ToplingTemp/Topling-243219-fH6SUi 2024/11/24-01:59:37.527485 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288072.sst): first pass time = 1.02's, 131.353'MB/sec 2024/11/24-01:59:38.859756 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288072.sst): old prealloc_size = 147639500, real_size = 32806944 2024/11/24-01:59:38.860284 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288072.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924518 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.18 avg-zval = 33.33 2024/11/24-01:59:38.861042 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288072.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:59:39.695030 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288072.sst): second pass time = 1.33's, 81.361'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.040'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3471.199'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 292.705'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33321.623'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.361'MB/sec zip pipeline throughput = 81.361'MB/sec entries = 924518 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.18 avg-zval = 33.33 usrkeys = 924518 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.18 avg-zval = 33.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924518 bytes = 3120256 } seq expand size = 6471626 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5124 value = 3.5153 dict = 1.00 all = 3.6489 } Zip/UnZip{ index = 0.0951 value = 0.2845 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 9.078225 GB avg = 0.167 KB (by entry num) total key len = 1.492595 GB avg = 0.028 KB total ukey len = 1.034859 GB avg = 0.015 KB total ukey num = 0.067399506 Billion total entry num = 0.054207235 Billion write speed all = 0.171974187 MB/sec (with seq num) write speed all = 0.164919109 MB/sec (without seq num) 2024/11/24-01:59:39.740960 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288072: 924518 keys, 36783224 bytes, temperature: kWarm 2024/11/24-01:59:39.741050 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384779741015, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288072, "file_size": 36783224, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4384238968, "table_properties": {"data_size": 30816720, "index_size": 1758912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25886504, "raw_average_key_size": 28, "raw_value_size": 108331312, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924518, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384776, "oldest_key_time": 0, "file_creation_time": 1732384776, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288072, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:59:39.779827 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288073.sst) /dev/shm/ToplingTemp/Topling-243219-iF9JAf 2024/11/24-01:59:40.788448 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288073.sst): first pass time = 1.01's, 133.075'MB/sec 2024/11/24-01:59:42.080472 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288073.sst): old prealloc_size = 147639500, real_size = 32986432 2024/11/24-01:59:42.081061 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288073.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925362 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.04 avg-zval = 33.49 2024/11/24-01:59:42.081810 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288073.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:59:42.924662 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288073.sst): second pass time = 1.29's, 83.879'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.788'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3106.359'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 293.509'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33959.639'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.879'MB/sec zip pipeline throughput = 83.879'MB/sec entries = 925362 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 33.49 usrkeys = 925362 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925362 bytes = 3123104 } seq expand size = 6477534 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5128 value = 3.4944 dict = 1.00 all = 3.6310 } Zip/UnZip{ index = 0.0951 value = 0.2862 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 9.193010 GB avg = 0.167 KB (by entry num) total key len = 1.518505 GB avg = 0.028 KB total ukey len = 1.053367 GB avg = 0.015 KB total ukey num = 0.068324868 Billion total entry num = 0.055132597 Billion write speed all = 0.174253972 MB/sec (with seq num) write speed all = 0.167078835 MB/sec (without seq num) 2024/11/24-01:59:42.970338 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288073: 925362 keys, 36964406 bytes, temperature: kWarm 2024/11/24-01:59:42.970408 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384782970373, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288073, "file_size": 36964406, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4384239344, "table_properties": {"data_size": 30994480, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25910136, "raw_average_key_size": 28, "raw_value_size": 108307712, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925362, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384779, "oldest_key_time": 0, "file_creation_time": 1732384779, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288073, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:59:43.006404 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288074.sst) /dev/shm/ToplingTemp/Topling-243219-9GPjEj 2024/11/24-01:59:43.937282 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288074.sst): first pass time = 0.93's, 144.191'MB/sec 2024/11/24-01:59:45.297197 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288074.sst): old prealloc_size = 147639500, real_size = 33152608 2024/11/24-01:59:45.297721 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288074.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926060 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.93 avg-zval = 33.65 2024/11/24-01:59:45.298503 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288074.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:59:46.149793 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288074.sst): second pass time = 1.36's, 79.680'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.554'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3461.740'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 295.867'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31165.417'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.680'MB/sec zip pipeline throughput = 79.680'MB/sec entries = 926060 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.93 avg-zval = 33.65 usrkeys = 926060 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.93 avg-zval = 33.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926060 bytes = 3125472 } seq expand size = 6482420 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5123 value = 3.4753 dict = 1.00 all = 3.6145 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 9.307781 GB avg = 0.166 KB (by entry num) total key len = 1.544435 GB avg = 0.028 KB total ukey len = 1.071888 GB avg = 0.015 KB total ukey num = 0.069250928 Billion total entry num = 0.056058657 Billion write speed all = 0.176533609 MB/sec (with seq num) write speed all = 0.169238334 MB/sec (without seq num) 2024/11/24-01:59:46.195643 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288074: 926060 keys, 37132686 bytes, temperature: kWarm 2024/11/24-01:59:46.195716 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384786195680, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288074, "file_size": 37132686, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4384239262, "table_properties": {"data_size": 31159056, "index_size": 1761856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25929680, "raw_average_key_size": 28, "raw_value_size": 108288172, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926060, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384782, "oldest_key_time": 0, "file_creation_time": 1732384782, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288074, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:59:46.233737 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288075.sst) /dev/shm/ToplingTemp/Topling-243219-YEeB5f 2024/11/24-01:59:47.249779 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288075.sst): first pass time = 1.02's, 132.104'MB/sec 2024/11/24-01:59:48.609656 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288075.sst): old prealloc_size = 147639500, real_size = 33471360 2024/11/24-01:59:48.610225 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288075.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926338 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.89 avg-zval = 33.97 2024/11/24-01:59:48.610995 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288075.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:59:49.458599 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288075.sst): second pass time = 1.36's, 79.675'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.117'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3270.837'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.12's, 301.914'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33676.727'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.675'MB/sec zip pipeline throughput = 79.675'MB/sec entries = 926338 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.89 avg-zval = 33.97 usrkeys = 926338 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.89 avg-zval = 33.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926338 bytes = 3126400 } seq expand size = 6484366 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4445 value = 3.4412 dict = 1.00 all = 3.5837 } Zip/UnZip{ index = 0.0957 value = 0.2906 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 9.422545 GB avg = 0.165 KB (by entry num) total key len = 1.570372 GB avg = 0.028 KB total ukey len = 1.090415 GB avg = 0.016 KB total ukey num = 0.070177266 Billion total entry num = 0.056984995 Billion write speed all = 0.178812794 MB/sec (with seq num) write speed all = 0.171397368 MB/sec (without seq num) 2024/11/24-01:59:49.505019 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288075: 926338 keys, 37452128 bytes, temperature: kWarm 2024/11/24-01:59:49.505091 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384789505055, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288075, "file_size": 37452128, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4384239459, "table_properties": {"data_size": 31465776, "index_size": 1773824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25937464, "raw_average_key_size": 28, "raw_value_size": 108280280, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926338, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384786, "oldest_key_time": 0, "file_creation_time": 1732384786, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288075, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:59:49.539041 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288076.sst) /dev/shm/ToplingTemp/Topling-243219-5G8aMj 2024/11/24-01:59:50.554928 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288076.sst): first pass time = 1.02's, 132.123'MB/sec 2024/11/24-01:59:51.915655 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288076.sst): old prealloc_size = 147639500, real_size = 33507200 2024/11/24-01:59:51.916248 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288076.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926091 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.93 avg-zval = 34.02 2024/11/24-01:59:51.916993 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288076.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:59:52.769630 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288076.sst): second pass time = 1.36's, 79.626'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.899'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3090.581'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 300.099'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32422.780'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.626'MB/sec zip pipeline throughput = 79.626'MB/sec entries = 926091 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.93 avg-zval = 34.02 usrkeys = 926091 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.93 avg-zval = 34.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926091 bytes = 3125568 } seq expand size = 6482637 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4455 value = 3.4374 dict = 1.00 all = 3.5803 } Zip/UnZip{ index = 0.0957 value = 0.2909 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 9.537315 GB avg = 0.165 KB (by entry num) total key len = 1.596303 GB avg = 0.028 KB total ukey len = 1.108936 GB avg = 0.016 KB total ukey num = 0.071103357 Billion total entry num = 0.057911086 Billion write speed all = 0.181091699 MB/sec (with seq num) write speed all = 0.173556167 MB/sec (without seq num) 2024/11/24-01:59:52.815623 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288076: 926091 keys, 37487738 bytes, temperature: kWarm 2024/11/24-01:59:52.815692 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384792815657, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288076, "file_size": 37487738, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4384240806, "table_properties": {"data_size": 31502320, "index_size": 1773184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25930548, "raw_average_key_size": 28, "raw_value_size": 108287218, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926091, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384789, "oldest_key_time": 0, "file_creation_time": 1732384789, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288076, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:59:52.851789 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288077.sst) /dev/shm/ToplingTemp/Topling-243219-BjxuLi 2024/11/24-01:59:53.873798 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288077.sst): first pass time = 1.02's, 131.332'MB/sec 2024/11/24-01:59:55.237025 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288077.sst): old prealloc_size = 147639500, real_size = 33638432 2024/11/24-01:59:55.237562 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288077.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926126 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.92 avg-zval = 34.17 2024/11/24-01:59:55.238295 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288077.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:59:56.088939 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288077.sst): second pass time = 1.36's, 79.483'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.158'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3429.383'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 304.399'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26030.883'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.483'MB/sec zip pipeline throughput = 79.483'MB/sec entries = 926126 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.92 avg-zval = 34.17 usrkeys = 926126 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.92 avg-zval = 34.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926126 bytes = 3125696 } seq expand size = 6482882 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5108 value = 3.4220 dict = 1.00 all = 3.5678 } Zip/UnZip{ index = 0.0951 value = 0.2922 dict = 1.00 all = 0.2803 } ---------------------------- total value len = 9.652084 GB avg = 0.164 KB (by entry num) total key len = 1.622234 GB avg = 0.028 KB total ukey len = 1.127459 GB avg = 0.016 KB total ukey num = 0.072029483 Billion total entry num = 0.058837212 Billion write speed all = 0.183370339 MB/sec (with seq num) write speed all = 0.175714710 MB/sec (without seq num) 2024/11/24-01:59:56.134996 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288077: 926126 keys, 37619042 bytes, temperature: kWarm 2024/11/24-01:59:56.135086 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384796135037, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288077, "file_size": 37619042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4384240379, "table_properties": {"data_size": 31644496, "index_size": 1762240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931528, "raw_average_key_size": 28, "raw_value_size": 108286276, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926126, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384792, "oldest_key_time": 0, "file_creation_time": 1732384792, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288077, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:59:56.172201 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288078.sst) /dev/shm/ToplingTemp/Topling-243219-HxWcXi 2024/11/24-01:59:57.144385 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288078.sst): first pass time = 0.97's, 138.063'MB/sec 2024/11/24-01:59:58.471187 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288078.sst): old prealloc_size = 147639500, real_size = 33588976 2024/11/24-01:59:58.471766 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288078.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925948 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.95 avg-zval = 34.12 2024/11/24-01:59:58.472508 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288078.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:59:59.324246 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288078.sst): second pass time = 1.33's, 81.668'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.042'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3168.728'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 303.640'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27149.539'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.668'MB/sec zip pipeline throughput = 81.668'MB/sec entries = 925948 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 34.12 usrkeys = 925948 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 34.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925948 bytes = 3125088 } seq expand size = 6481636 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5160 value = 3.4273 dict = 1.00 all = 3.5725 } Zip/UnZip{ index = 0.0951 value = 0.2918 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 9.766857 GB avg = 0.163 KB (by entry num) total key len = 1.648161 GB avg = 0.028 KB total ukey len = 1.145978 GB avg = 0.016 KB total ukey num = 0.072955431 Billion total entry num = 0.059763160 Billion write speed all = 0.185648965 MB/sec (with seq num) write speed all = 0.177873266 MB/sec (without seq num) 2024/11/24-01:59:59.370622 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288078: 925948 keys, 37569196 bytes, temperature: kWarm 2024/11/24-01:59:59.370689 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384799370655, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288078, "file_size": 37569196, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4384240694, "table_properties": {"data_size": 31596256, "index_size": 1761024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926544, "raw_average_key_size": 28, "raw_value_size": 108291252, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925948, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384796, "oldest_key_time": 0, "file_creation_time": 1732384796, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288078, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:59:59.411172 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288079.sst) /dev/shm/ToplingTemp/Topling-243219-Li7EXg 2024/11/24-02:00:00.491572 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288079.sst): first pass time = 1.08's, 124.234'MB/sec 2024/11/24-02:00:01.830741 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288079.sst): old prealloc_size = 147639500, real_size = 33756144 2024/11/24-02:00:01.831305 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288079.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926160 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0318 GB avg-val = 116.92 avg-zval = 34.28 2024/11/24-02:00:01.832067 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288079.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:00:02.682335 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288079.sst): second pass time = 1.34's, 80.956'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.319'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3269.364'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 296.410'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33229.544'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.956'MB/sec zip pipeline throughput = 80.956'MB/sec entries = 926160 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.92 avg-zval = 34.28 usrkeys = 926160 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.92 avg-zval = 34.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926160 bytes = 3125808 } seq expand size = 6483120 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4440 value = 3.4105 dict = 1.00 all = 3.5563 } Zip/UnZip{ index = 0.0957 value = 0.2932 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 9.881626 GB avg = 0.163 KB (by entry num) total key len = 1.674093 GB avg = 0.028 KB total ukey len = 1.164501 GB avg = 0.016 KB total ukey num = 0.073881591 Billion total entry num = 0.060689320 Billion write speed all = 0.187927000 MB/sec (with seq num) write speed all = 0.180031231 MB/sec (without seq num) 2024/11/24-02:00:02.730083 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288079: 926160 keys, 37740642 bytes, temperature: kWarm 2024/11/24-02:00:02.730155 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384802730118, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288079, "file_size": 37740642, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4384239072, "table_properties": {"data_size": 31750880, "index_size": 1773568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25932480, "raw_average_key_size": 28, "raw_value_size": 108285268, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926160, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384799, "oldest_key_time": 0, "file_creation_time": 1732384799, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288079, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:00:02.765616 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288080.sst) /dev/shm/ToplingTemp/Topling-243219-VJLfvi 2024/11/24-02:00:03.733642 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288080.sst): first pass time = 0.97's, 138.659'MB/sec 2024/11/24-02:00:05.005492 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288080.sst): old prealloc_size = 147639500, real_size = 27912016 2024/11/24-02:00:05.006120 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288080.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 944828 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0259 GB avg-val = 114.06 avg-zval = 27.38 2024/11/24-02:00:05.007882 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288080.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:00:05.851422 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288080.sst): second pass time = 1.27's, 84.816'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 37.761'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3001.779'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.329'MB/sec write SST data time = 0.13's, 250.865'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.074'MB, throughput = 32067.923'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.816'MB/sec zip pipeline throughput = 84.816'MB/sec entries = 944828 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.06 avg-zval = 27.38 usrkeys = 944828 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.06 avg-zval = 27.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 944828 bytes = 3188816 } seq expand size = 6613796 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0259 GB dict = 1.07 MB all = 0.0319 GB } UnZip/Zip{ index = 10.4442 value = 4.1661 dict = 1.00 all = 4.2019 } Zip/UnZip{ index = 0.0957 value = 0.2400 dict = 1.00 all = 0.2380 } ---------------------------- total value len = 9.996002 GB avg = 0.162 KB (by entry num) total key len = 1.700549 GB avg = 0.028 KB total ukey len = 1.183398 GB avg = 0.016 KB total ukey num = 0.074826419 Billion total entry num = 0.061634148 Billion write speed all = 0.190207498 MB/sec (with seq num) write speed all = 0.182189218 MB/sec (without seq num) 2024/11/24-02:00:05.892927 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288080: 944828 keys, 31942290 bytes, temperature: kWarm 2024/11/24-02:00:05.893018 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384805892963, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288080, "file_size": 31942290, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4384239504, "table_properties": {"data_size": 25866368, "index_size": 1809280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26455184, "raw_average_key_size": 28, "raw_value_size": 107762672, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 944828, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384802, "oldest_key_time": 0, "file_creation_time": 1732384802, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288080, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:00:05.941414 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288081.sst) /dev/shm/ToplingTemp/Topling-243219-FNDRCj 2024/11/24-02:00:06.852363 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288081.sst): first pass time = 0.91's, 147.346'MB/sec 2024/11/24-02:00:08.179004 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288081.sst): old prealloc_size = 147639500, real_size = 32828544 2024/11/24-02:00:08.179525 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288081.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924357 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.20 avg-zval = 33.36 2024/11/24-02:00:08.180315 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288081.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:00:09.016079 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288081.sst): second pass time = 1.33's, 81.738'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.968'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3575.702'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 290.979'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36040.259'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.738'MB/sec zip pipeline throughput = 81.738'MB/sec entries = 924357 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.20 avg-zval = 33.36 usrkeys = 924357 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.20 avg-zval = 33.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924357 bytes = 3119712 } seq expand size = 6470499 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5113 value = 3.5130 dict = 1.00 all = 3.6468 } Zip/UnZip{ index = 0.0951 value = 0.2847 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 10.110809 GB avg = 0.162 KB (by entry num) total key len = 1.726431 GB avg = 0.028 KB total ukey len = 1.201885 GB avg = 0.016 KB total ukey num = 0.075750776 Billion total entry num = 0.062558505 Billion write speed all = 0.192485444 MB/sec (with seq num) write speed all = 0.184347330 MB/sec (without seq num) 2024/11/24-02:00:09.063607 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288081: 924357 keys, 36804448 bytes, temperature: kWarm 2024/11/24-02:00:09.063678 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384809063643, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288081, "file_size": 36804448, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4384238864, "table_properties": {"data_size": 30838512, "index_size": 1758784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25881996, "raw_average_key_size": 28, "raw_value_size": 108335866, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924357, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384805, "oldest_key_time": 0, "file_creation_time": 1732384805, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288081, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:00:09.102644 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288082.sst) /dev/shm/ToplingTemp/Topling-243219-edtGYi 2024/11/24-02:00:09.931153 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288082.sst): first pass time = 0.83's, 162.007'MB/sec 2024/11/24-02:00:11.185038 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288082.sst): old prealloc_size = 147639500, real_size = 28304288 2024/11/24-02:00:11.185569 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288082.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 943852 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0263 GB avg-val = 114.20 avg-zval = 27.84 2024/11/24-02:00:11.187281 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288082.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:00:12.015485 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288082.sst): second pass time = 1.25's, 86.020'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 38.936'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3535.930'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.13's, 252.545'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.074'MB, throughput = 33784.433'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.020'MB/sec zip pipeline throughput = 86.020'MB/sec entries = 943852 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.20 avg-zval = 27.84 usrkeys = 943852 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.20 avg-zval = 27.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 943852 bytes = 3185520 } seq expand size = 6606964 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0263 GB dict = 1.07 MB all = 0.0323 GB } UnZip/Zip{ index = 10.5130 value = 4.1028 dict = 1.00 all = 4.1510 } Zip/UnZip{ index = 0.0951 value = 0.2437 dict = 1.00 all = 0.2409 } ---------------------------- total value len = 10.225205 GB avg = 0.161 KB (by entry num) total key len = 1.752858 GB avg = 0.028 KB total ukey len = 1.220762 GB avg = 0.016 KB total ukey num = 0.076694628 Billion total entry num = 0.063502357 Billion write speed all = 0.194765897 MB/sec (with seq num) write speed all = 0.186505401 MB/sec (without seq num) 2024/11/24-02:00:12.056290 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288082: 943852 keys, 32333536 bytes, temperature: kWarm 2024/11/24-02:00:12.056361 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384812056326, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288082, "file_size": 32333536, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4384240321, "table_properties": {"data_size": 26272528, "index_size": 1795584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26427856, "raw_average_key_size": 28, "raw_value_size": 107789888, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 943852, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384809, "oldest_key_time": 0, "file_creation_time": 1732384809, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288082, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:00:12.090505 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288083.sst) /dev/shm/ToplingTemp/Topling-243219-l40KJg 2024/11/24-02:00:12.969899 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288083.sst): first pass time = 0.88's, 152.632'MB/sec 2024/11/24-02:00:14.272238 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288083.sst): old prealloc_size = 147639500, real_size = 31371936 2024/11/24-02:00:14.272776 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288083.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.96 2024/11/24-02:00:14.273542 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288083.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:00:15.083416 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288083.sst): second pass time = 1.30's, 83.347'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.560'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3415.566'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 279.863'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34923.716'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.347'MB/sec zip pipeline throughput = 83.347'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.96 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4466 value = 3.6920 dict = 1.00 all = 3.7986 } Zip/UnZip{ index = 0.0957 value = 0.2709 dict = 1.00 all = 0.2633 } ---------------------------- total value len = 10.340118 GB avg = 0.161 KB (by entry num) total key len = 1.778599 GB avg = 0.028 KB total ukey len = 1.239148 GB avg = 0.016 KB total ukey num = 0.077613928 Billion total entry num = 0.064421657 Billion write speed all = 0.197043114 MB/sec (with seq num) write speed all = 0.188663452 MB/sec (without seq num) 2024/11/24-02:00:15.128381 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288083: 919300 keys, 35333746 bytes, temperature: kWarm 2024/11/24-02:00:15.128451 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384815128415, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288083, "file_size": 35333746, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4267917588, "largest_seqno": 4383773342, "table_properties": {"data_size": 29381904, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384812, "oldest_key_time": 0, "file_creation_time": 1732384812, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288083, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:00:15.167736 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288084.sst) /dev/shm/ToplingTemp/Topling-243219-XRJtof 2024/11/24-02:00:16.177058 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288084.sst): first pass time = 1.01's, 132.983'MB/sec 2024/11/24-02:00:17.516061 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288084.sst): old prealloc_size = 147639500, real_size = 32525856 2024/11/24-02:00:17.516621 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288084.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933332 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.80 avg-zval = 32.68 2024/11/24-02:00:17.517378 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288084.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:00:18.374528 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288084.sst): second pass time = 1.34's, 80.808'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.550'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3307.123'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 288.058'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33412.330'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.808'MB/sec zip pipeline throughput = 80.808'MB/sec entries = 933332 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.80 avg-zval = 32.68 usrkeys = 933332 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.80 avg-zval = 32.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933332 bytes = 3150016 } seq expand size = 6533324 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4439 value = 3.5432 dict = 1.00 all = 3.6745 } Zip/UnZip{ index = 0.0957 value = 0.2822 dict = 1.00 all = 0.2721 } ---------------------------- total value len = 10.454736 GB avg = 0.160 KB (by entry num) total key len = 1.804732 GB avg = 0.028 KB total ukey len = 1.257815 GB avg = 0.016 KB total ukey num = 0.078547260 Billion total entry num = 0.065354989 Billion write speed all = 0.199320976 MB/sec (with seq num) write speed all = 0.190820366 MB/sec (without seq num) 2024/11/24-02:00:18.419668 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288084: 933332 keys, 36526446 bytes, temperature: kWarm 2024/11/24-02:00:18.419743 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384818419702, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288084, "file_size": 36526446, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4384240479, "table_properties": {"data_size": 30505040, "index_size": 1787328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26133296, "raw_average_key_size": 28, "raw_value_size": 108084460, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933332, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384815, "oldest_key_time": 0, "file_creation_time": 1732384815, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288084, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:00:18.460168 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288085.sst) /dev/shm/ToplingTemp/Topling-243219-NDAWti 2024/11/24-02:00:19.478518 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288085.sst): first pass time = 1.02's, 131.803'MB/sec 2024/11/24-02:00:20.765892 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288085.sst): old prealloc_size = 147639500, real_size = 30390064 2024/11/24-02:00:20.766470 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288085.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928850 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.50 avg-zval = 30.55 2024/11/24-02:00:20.767232 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288085.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:00:21.579553 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288085.sst): second pass time = 1.29's, 84.110'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.250'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3206.346'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.339'MB/sec write SST data time = 0.13's, 270.248'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33258.864'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.110'MB/sec zip pipeline throughput = 84.110'MB/sec entries = 928850 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 30.55 usrkeys = 928850 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 30.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928850 bytes = 3134880 } seq expand size = 6501950 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.4450 value = 3.8130 dict = 1.00 all = 3.9044 } Zip/UnZip{ index = 0.0957 value = 0.2623 dict = 1.00 all = 0.2561 } ---------------------------- total value len = 10.569448 GB avg = 0.159 KB (by entry num) total key len = 1.830740 GB avg = 0.028 KB total ukey len = 1.276392 GB avg = 0.016 KB total ukey num = 0.079476110 Billion total entry num = 0.066283839 Billion write speed all = 0.201598368 MB/sec (with seq num) write speed all = 0.192977393 MB/sec (without seq num) 2024/11/24-02:00:21.622213 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288085: 928850 keys, 34376336 bytes, temperature: kWarm 2024/11/24-02:00:21.622288 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384821622251, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288085, "file_size": 34376336, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4314231674, "largest_seqno": 4384239549, "table_properties": {"data_size": 28379104, "index_size": 1778560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007800, "raw_average_key_size": 28, "raw_value_size": 108209988, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928850, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384818, "oldest_key_time": 0, "file_creation_time": 1732384818, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288085, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:00:21.658760 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288086.sst) /dev/shm/ToplingTemp/Topling-243219-stBHGg 2024/11/24-02:00:21.812933 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288086.sst): first pass time = 0.15's, 237.960'MB/sec 2024/11/24-02:00:22.175040 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288086.sst): old prealloc_size = 147639500, real_size = 10483584 2024/11/24-02:00:22.175241 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288086.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 252844 prefix = 4 raw-key = 0.0040 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0296 GB zip-val = 0.0099 GB avg-val = 117.06 avg-zval = 39.30 2024/11/24-02:00:22.175796 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288086.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.037 GB 2024/11/24-02:00:22.398590 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00/288086.sst): second pass time = 0.36's, 82.026'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 51.788'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2690.609'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.337'MB/sec write SST data time = 0.04's, 322.238'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.302'MB, throughput = 10835.314'MB/sec zip my value time = 0.36's, unzip length = 0.030'GB zip my value throughput = 82.026'MB/sec zip pipeline throughput = 82.026'MB/sec entries = 252844 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.06 avg-zval = 39.30 usrkeys = 252844 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.06 avg-zval = 39.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 252844 bytes = 853360 } seq expand size = 1769908 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0051 GB value = 0.0296 GB dict = 0.30 MB all = 0.0367 GB } __ZipSize{ index = 0.0005 GB value = 0.0099 GB dict = 0.30 MB all = 0.0116 GB } UnZip/Zip{ index = 10.4876 value = 2.9783 dict = 1.00 all = 3.1679 } Zip/UnZip{ index = 0.0954 value = 0.3358 dict = 1.00 all = 0.3157 } ---------------------------- total value len = 10.600816 GB avg = 0.159 KB (by entry num) total key len = 1.837820 GB avg = 0.028 KB total ukey len = 1.281448 GB avg = 0.016 KB total ukey num = 0.079728954 Billion total entry num = 0.066536683 Billion write speed all = 0.202220749 MB/sec (with seq num) write speed all = 0.193567004 MB/sec (without seq num) 2024/11/24-02:00:22.419665 139957458552384 [compaction_job.cc:1948] [default] [JOB 39738] Generated table #288086: 252844 keys, 11577971 bytes, temperature: kWarm 2024/11/24-02:00:22.419724 139957458552384 EVENT_LOG_v1 {"time_micros": 1732384822419692, "cf_name": "default", "job": 39738, "event": "table_file_creation", "file_number": 288086, "file_size": 11577971, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4384239890, "table_properties": {"data_size": 9938032, "index_size": 482176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7079632, "raw_average_key_size": 28, "raw_value_size": 29598360, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 252844, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384821, "oldest_key_time": 0, "file_creation_time": 1732384821, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288086, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:00:22.433063 139957458552384 [compaction_job.cc:703] job-39738: subcompact[0], size: 0.553951 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39738/att-00 [288071,288072,288073,288074,288075,288076,288077,288078,288079,288080,288081,288082,288083,288084,288085,288086] 2024/11/24-02:00:22.527472 139957458552384 (Original Log Time 2024/11/24-01:59:32.966594) EVENT_LOG_v1 {"time_micros": 1732384772966554, "job": 39738, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [288022], "files_L3": [288067, 288068, 287979, 287980, 287981, 287982, 287983, 287984, 287985, 287986, 287957, 287958, 287959, 287960, 287961, 287962, 288044], "score": 0.769222, "input_data_size": 577557558, "oldest_snapshot_seqno": 4384613912} 2024/11/24-02:00:22.527476 139957458552384 (Original Log Time 2024/11/24-02:00:22.526210) [compaction_job.cc:2022] [default] [JOB 39738] Compacted 1@2 + 17@3 files to L3 => 553950559 bytes 2024/11/24-02:00:22.527478 139957458552384 (Original Log Time 2024/11/24-02:00:22.527409) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 11.7 rd, 11.2 wr, level 3, files in(1, 17) out(16 +0 blob) MB in(27.1, 523.7 +0.0 blob) out(528.3 +0.0 blob), read-write-amplify(39.8) write-amplify(19.5) OK, records in: 14282167, records dropped: 103648 output_compression: Snappy 2024/11/24-02:00:22.527483 139957458552384 (Original Log Time 2024/11/24-02:00:22.527452) EVENT_LOG_v1 {"time_micros": 1732384822527433, "cf": "default", "job": 39738, "event": "compaction_finished", "compaction_time_micros": 49466476, "compaction_time_cpu_micros": 21152754, "output_level": 3, "num_output_files": 16, "total_output_size": 553950559, "num_input_records": 14282167, "num_output_records": 14178519, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 379338346, "file_range_sync_nanos": 0, "file_fsync_nanos": 11078, "file_prepare_write_nanos": 39391, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/24-02:00:22.660342 139957458552384 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732384772,"executesMs":49696,"compactionJobId":39738,"attempt":0,"compactionInputRawBytes":2064662994,"compactionInputZipBytes":577557558,"compactionOutputRawBytes":0,"compactionOutputZipBytes":553950559,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-7bd4064389c89ad51e64aabeb008a9fd-f6c42677d9535d39-00"}}, response = 2024/11/24-02:00:22.660428 139957458552384 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39738/att-00: olev 3, work 49.697 s, result 7.798 ms, install 1.289 ms, input{raw 1.923 GiB zip 550.802 MiB}