2024/11/23-19:32:36.003043 139957248734784 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00: fork to child time = 0.008422 sec 2024/11/23-19:32:36.003742 139957248734784 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00 2024/11/23-19:32:36.010653 139957248734784 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00 2024/11/23-19:32:36.011524 139957248734784 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-19:32:36.011720 139957248734784 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-19:32:36.067627 139957248734784 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34283/att-00/MANIFEST-206347 2024/11/23-19:32:36.084559 139957248734784 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34283/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 277361, last_sequence is 4217972598, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 277196 2024/11/23-19:32:36.084566 139957248734784 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-19:32:36.084750 139957248734784 [version_set.cc:5721] Creating manifest 277361 2024/11/23-19:32:36.120303 139957248734784 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-19:32:36.141345 139957248734784 [compaction_job.cc:2437] [default] [JOB 34283] Compacting 1@2 + 20@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-19:32:36.141369 139957248734784 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 25661 Base level 2, inputs: [277333(27MB)], [277352(37MB) 277353(4368KB) 277279(38MB) 277280(37MB) 277281(35MB) 277282(37MB) 277283(34MB) 277284(35MB) 277285(37MB) 277286(35MB) 277287(37MB) 277288(35MB) 277289(35MB) 277290(36MB) 277291(36MB) 277303(37MB) 277304(36MB) 277305(36MB) 277306(34MB) 277354(33MB)] 2024/11/23-19:32:36.147999 139957248734784 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:32:36.148022 139957248734784 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:32:36.148037 139957248734784 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:32:36.148042 139957248734784 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:32:36.148052 139957248734784 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:32:36.818704 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277362.sst) /dev/shm/ToplingTemp/Topling-166939-ALLJUj 2024/11/23-19:32:37.495784 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277362.sst): first pass time = 0.68's, 198.247'MB/sec 2024/11/23-19:32:39.455424 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277362.sst): old prealloc_size = 147639500, real_size = 35093824 2024/11/23-19:32:39.456003 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277362.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931261 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0331 GB avg-val = 116.12 avg-zval = 35.53 2024/11/23-19:32:39.456723 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277362.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:32:40.287568 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277362.sst): second pass time = 1.96's, 55.223'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 46.694'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3169.978'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.330'MB/sec write SST data time = 0.13's, 307.505'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31501.884'MB/sec zip my value time = 1.96's, unzip length = 0.108'GB zip my value throughput = 55.223'MB/sec zip pipeline throughput = 138.681'MB/sec entries = 931261 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 35.53 usrkeys = 931261 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 35.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931261 bytes = 2910208 } seq expand size = 6518827 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.5118 value = 3.2682 dict = 1.00 all = 3.4543 } Zip/UnZip{ index = 0.0951 value = 0.3060 dict = 1.00 all = 0.2895 } ---------------------------- total value len = 8.963281 GB avg = 0.168 KB (by entry num) total key len = 1.466896 GB avg = 0.028 KB total ukey len = 1.016503 GB avg = 0.015 KB total ukey num = 0.066481696 Billion total entry num = 0.053289425 Billion write speed all = 0.272698118 MB/sec (with seq num) write speed all = 0.261552056 MB/sec (without seq num) 2024/11/23-19:32:40.336411 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277362: 931261 keys, 38855760 bytes, temperature: kWarm 2024/11/23-19:32:40.336503 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361560336449, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277362, "file_size": 38855760, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4191256266, "largest_seqno": 4217722457, "table_properties": {"data_size": 33089008, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26075308, "raw_average_key_size": 28, "raw_value_size": 108142486, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931261, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361556, "oldest_key_time": 0, "file_creation_time": 1732361556, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277362, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:32:40.397439 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277363.sst) /dev/shm/ToplingTemp/Topling-166939-WOUtNl 2024/11/23-19:32:41.691013 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277363.sst): first pass time = 1.29's, 103.762'MB/sec 2024/11/23-19:32:43.439186 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277363.sst): old prealloc_size = 147639500, real_size = 36591040 2024/11/23-19:32:43.439741 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277363.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936106 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0346 GB avg-val = 115.38 avg-zval = 36.91 2024/11/23-19:32:43.440589 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277363.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:32:44.457426 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277363.sst): second pass time = 1.75's, 61.810'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 40.275'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3455.424'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.279'MB/sec write SST data time = 0.18's, 229.658'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34883.843'MB/sec zip my value time = 1.75's, unzip length = 0.108'GB zip my value throughput = 61.810'MB/sec zip pipeline throughput = 105.876'MB/sec entries = 936106 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.38 avg-zval = 36.91 usrkeys = 936106 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.20 avg-zval = 36.91 TagRS{ kinds = RS_Key0_TagN bytes = 146360 } TagArray{ size = 911634 bytes = 3418640 } seq expand size = 6381438 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0410 GB } UnZip/Zip{ index = 10.3772 value = 3.1259 dict = 1.00 all = 3.2729 } Zip/UnZip{ index = 0.0964 value = 0.3199 dict = 1.00 all = 0.3055 } ---------------------------- total value len = 9.077669 GB avg = 0.167 KB (by entry num) total key len = 1.493107 GB avg = 0.028 KB total ukey len = 1.035225 GB avg = 0.015 KB total ukey num = 0.067417802 Billion total entry num = 0.054225531 Billion write speed all = 0.276343972 MB/sec (with seq num) write speed all = 0.265003349 MB/sec (without seq num) 2024/11/23-19:32:44.506421 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277363: 936106 keys, 41008662 bytes, temperature: kWarm 2024/11/23-19:32:44.506488 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361564506453, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277363, "file_size": 41008662, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4217722470, "table_properties": {"data_size": 34552688, "index_size": 1804160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26210968, "raw_average_key_size": 28, "raw_value_size": 108006808, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936106, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361560, "oldest_key_time": 0, "file_creation_time": 1732361560, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277363, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:32:44.550312 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277364.sst) /dev/shm/ToplingTemp/Topling-166939-noVhpn 2024/11/23-19:32:45.462095 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277364.sst): first pass time = 0.91's, 147.210'MB/sec 2024/11/23-19:32:47.023092 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277364.sst): old prealloc_size = 147639500, real_size = 35748736 2024/11/23-19:32:47.023629 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277364.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930192 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 116.29 avg-zval = 36.25 2024/11/23-19:32:47.024266 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277364.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:32:47.991286 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277364.sst): second pass time = 1.56's, 69.335'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 40.467'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3467.263'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.13's, 304.861'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34550.620'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.335'MB/sec zip pipeline throughput = 69.335'MB/sec entries = 930192 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.29 avg-zval = 36.25 usrkeys = 930192 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.17 avg-zval = 36.25 TagRS{ kinds = RS_Key0_TagN bytes = 145440 } TagArray{ size = 647872 bytes = 2429536 } seq expand size = 4535104 multi value expand size = 0 cnt WriteAppend = 20 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.3753 value = 3.2077 dict = 1.00 all = 3.4257 } Zip/UnZip{ index = 0.0964 value = 0.3118 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 9.190377 GB avg = 0.167 KB (by entry num) total key len = 1.519153 GB avg = 0.028 KB total ukey len = 1.053829 GB avg = 0.015 KB total ukey num = 0.068347994 Billion total entry num = 0.055155723 Billion write speed all = 0.279945425 MB/sec (with seq num) write speed all = 0.268411330 MB/sec (without seq num) 2024/11/23-19:32:48.042337 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277364: 930192 keys, 39179590 bytes, temperature: kWarm 2024/11/23-19:32:48.042400 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361568042366, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277364, "file_size": 39179590, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775967, "table_properties": {"data_size": 33722928, "index_size": 1793088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26045376, "raw_average_key_size": 28, "raw_value_size": 108172480, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930192, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361564, "oldest_key_time": 0, "file_creation_time": 1732361564, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277364, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:32:48.079255 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277365.sst) /dev/shm/ToplingTemp/Topling-166939-K2lM0l 2024/11/23-19:32:49.059750 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277365.sst): first pass time = 0.98's, 136.893'MB/sec 2024/11/23-19:32:50.559806 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277365.sst): old prealloc_size = 147639500, real_size = 34316944 2024/11/23-19:32:50.560538 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277365.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925453 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0323 GB avg-val = 117.03 avg-zval = 34.93 2024/11/23-19:32:50.561431 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277365.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:32:51.596036 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277365.sst): second pass time = 1.50's, 72.240'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 37.181'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2482.391'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.264'MB/sec write SST data time = 0.16's, 243.764'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33122.766'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.240'MB/sec zip pipeline throughput = 72.240'MB/sec entries = 925453 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.03 avg-zval = 34.93 usrkeys = 925453 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.03 avg-zval = 34.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925453 bytes = 3354784 } seq expand size = 6478171 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5150 value = 3.3505 dict = 1.00 all = 3.4838 } Zip/UnZip{ index = 0.0951 value = 0.2985 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 9.305160 GB avg = 0.166 KB (by entry num) total key len = 1.545065 GB avg = 0.028 KB total ukey len = 1.072338 GB avg = 0.015 KB total ukey num = 0.069273447 Billion total entry num = 0.056081176 Billion write speed all = 0.283596475 MB/sec (with seq num) write speed all = 0.271869955 MB/sec (without seq num) 2024/11/23-19:32:51.644957 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277365: 925453 keys, 38526724 bytes, temperature: kWarm 2024/11/23-19:32:51.645046 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361571645007, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277365, "file_size": 38526724, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012776021, "table_properties": {"data_size": 32325120, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912684, "raw_average_key_size": 28, "raw_value_size": 108305174, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925453, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361568, "oldest_key_time": 0, "file_creation_time": 1732361568, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277365, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:32:51.685119 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277366.sst) /dev/shm/ToplingTemp/Topling-166939-zIOpCl 2024/11/23-19:32:52.928219 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277366.sst): first pass time = 1.24's, 107.974'MB/sec 2024/11/23-19:32:54.378362 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277366.sst): old prealloc_size = 147639500, real_size = 36179552 2024/11/23-19:32:54.379079 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277366.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926493 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0342 GB avg-val = 116.87 avg-zval = 36.89 2024/11/23-19:32:54.379878 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277366.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:32:55.365219 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277366.sst): second pass time = 1.45's, 74.715'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 40.504'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2547.289'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.16's, 257.029'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27517.477'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.715'MB/sec zip pipeline throughput = 74.715'MB/sec entries = 926493 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.87 avg-zval = 36.89 usrkeys = 926493 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.48 avg-zval = 36.89 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 743496 bytes = 2788128 } seq expand size = 5204472 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4451 value = 3.1684 dict = 1.00 all = 3.3579 } Zip/UnZip{ index = 0.0957 value = 0.3156 dict = 1.00 all = 0.2978 } ---------------------------- total value len = 9.418641 GB avg = 0.165 KB (by entry num) total key len = 1.571007 GB avg = 0.028 KB total ukey len = 1.090868 GB avg = 0.016 KB total ukey num = 0.070199940 Billion total entry num = 0.057007669 Billion write speed all = 0.287212313 MB/sec (with seq num) write speed all = 0.275293239 MB/sec (without seq num) 2024/11/23-19:32:55.415247 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277366: 926493 keys, 39970628 bytes, temperature: kWarm 2024/11/23-19:32:55.415321 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361575415283, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277366, "file_size": 39970628, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 34173712, "index_size": 1774016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941804, "raw_average_key_size": 28, "raw_value_size": 108276018, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926493, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361571, "oldest_key_time": 0, "file_creation_time": 1732361571, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277366, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:32:55.456842 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277367.sst) /dev/shm/ToplingTemp/Topling-166939-nuLe6j 2024/11/23-19:32:56.441642 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277367.sst): first pass time = 0.98's, 136.296'MB/sec 2024/11/23-19:32:57.792171 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277367.sst): old prealloc_size = 147639500, real_size = 32938576 2024/11/23-19:32:57.792721 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277367.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930732 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.21 avg-zval = 33.24 2024/11/23-19:32:57.793569 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277367.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:32:58.861048 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277367.sst): second pass time = 1.35's, 80.134'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 34.771'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3436.351'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.23's, 158.717'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29376.079'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.134'MB/sec zip pipeline throughput = 80.134'MB/sec entries = 930732 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.21 avg-zval = 33.24 usrkeys = 930732 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.21 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930732 bytes = 3373920 } seq expand size = 6515124 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5142 value = 3.4962 dict = 1.00 all = 3.6115 } Zip/UnZip{ index = 0.0951 value = 0.2860 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 9.533313 GB avg = 0.165 KB (by entry num) total key len = 1.597068 GB avg = 0.028 KB total ukey len = 1.109483 GB avg = 0.016 KB total ukey num = 0.071130672 Billion total entry num = 0.057938401 Billion write speed all = 0.290863767 MB/sec (with seq num) write speed all = 0.278751203 MB/sec (without seq num) 2024/11/23-19:32:58.911936 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277367: 930732 keys, 37164132 bytes, temperature: kWarm 2024/11/23-19:32:58.912044 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361578911993, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277367, "file_size": 37164132, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 30935296, "index_size": 1770432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060496, "raw_average_key_size": 28, "raw_value_size": 108157320, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930732, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361575, "oldest_key_time": 0, "file_creation_time": 1732361575, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277367, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:32:58.955697 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277368.sst) /dev/shm/ToplingTemp/Topling-166939-qdnJBk 2024/11/23-19:32:59.905867 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277368.sst): first pass time = 0.95's, 141.262'MB/sec 2024/11/23-19:33:01.227744 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277368.sst): old prealloc_size = 147639500, real_size = 32041312 2024/11/23-19:33:01.228362 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277368.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936596 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0300 GB avg-val = 115.30 avg-zval = 32.05 2024/11/23-19:33:01.229166 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277368.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:02.093442 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277368.sst): second pass time = 1.32's, 81.747'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.915'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3016.497'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.318'MB/sec write SST data time = 0.13's, 280.025'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34187.673'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.747'MB/sec zip pipeline throughput = 81.747'MB/sec entries = 936596 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.30 avg-zval = 32.05 usrkeys = 936596 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.30 avg-zval = 32.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936596 bytes = 3395168 } seq expand size = 6556172 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4438 value = 3.5982 dict = 1.00 all = 3.6990 } Zip/UnZip{ index = 0.0958 value = 0.2779 dict = 1.00 all = 0.2703 } ---------------------------- total value len = 9.647863 GB avg = 0.164 KB (by entry num) total key len = 1.623292 GB avg = 0.028 KB total ukey len = 1.128215 GB avg = 0.016 KB total ukey num = 0.072067268 Billion total entry num = 0.058874997 Billion write speed all = 0.294517654 MB/sec (with seq num) write speed all = 0.282210325 MB/sec (without seq num) 2024/11/23-19:33:02.138572 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277368: 936596 keys, 36284812 bytes, temperature: kWarm 2024/11/23-19:33:02.138649 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361582138610, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277368, "file_size": 36284812, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30013392, "index_size": 1793600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26224688, "raw_average_key_size": 28, "raw_value_size": 107993132, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936596, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361578, "oldest_key_time": 0, "file_creation_time": 1732361578, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277368, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:02.178036 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277369.sst) /dev/shm/ToplingTemp/Topling-166939-OrLwCm 2024/11/23-19:33:03.232393 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277369.sst): first pass time = 1.05's, 127.305'MB/sec 2024/11/23-19:33:04.598253 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277369.sst): old prealloc_size = 147639500, real_size = 35329392 2024/11/23-19:33:04.598787 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277369.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927409 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 116.72 avg-zval = 35.93 2024/11/23-19:33:04.600395 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277369.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:05.439431 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277369.sst): second pass time = 1.36's, 79.328'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.151'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3488.297'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 306.876'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25819.933'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.328'MB/sec zip pipeline throughput = 79.328'MB/sec entries = 927409 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.72 avg-zval = 35.93 usrkeys = 927409 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.72 avg-zval = 35.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927409 bytes = 3477792 } seq expand size = 6491863 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.4438 value = 3.2487 dict = 1.00 all = 3.3841 } Zip/UnZip{ index = 0.0958 value = 0.3078 dict = 1.00 all = 0.2955 } ---------------------------- total value len = 9.762605 GB avg = 0.163 KB (by entry num) total key len = 1.649260 GB avg = 0.028 KB total ukey len = 1.146763 GB avg = 0.016 KB total ukey num = 0.072994677 Billion total entry num = 0.059802406 Billion write speed all = 0.298168357 MB/sec (with seq num) write speed all = 0.285668254 MB/sec (without seq num) 2024/11/23-19:33:05.488318 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277369: 927409 keys, 39661442 bytes, temperature: kWarm 2024/11/23-19:33:05.488393 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361585488355, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277369, "file_size": 39661442, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33321376, "index_size": 1776000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25967452, "raw_average_key_size": 28, "raw_value_size": 108250326, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927409, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361582, "oldest_key_time": 0, "file_creation_time": 1732361582, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277369, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:05.527900 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277370.sst) /dev/shm/ToplingTemp/Topling-166939-fAYu1k 2024/11/23-19:33:06.556278 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277370.sst): first pass time = 1.03's, 130.520'MB/sec 2024/11/23-19:33:07.890508 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277370.sst): old prealloc_size = 147639500, real_size = 31709248 2024/11/23-19:33:07.891092 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277370.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0297 GB avg-val = 116.13 avg-zval = 31.90 2024/11/23-19:33:07.892601 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277370.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:08.732410 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277370.sst): second pass time = 1.33's, 81.105'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.686'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3152.859'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 279.840'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34096.936'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.105'MB/sec zip pipeline throughput = 81.105'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 31.90 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 31.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5127 value = 3.6406 dict = 1.00 all = 3.7349 } Zip/UnZip{ index = 0.0951 value = 0.2747 dict = 1.00 all = 0.2677 } ---------------------------- total value len = 9.877266 GB avg = 0.163 KB (by entry num) total key len = 1.675335 GB avg = 0.028 KB total ukey len = 1.165388 GB avg = 0.016 KB total ukey num = 0.073925919 Billion total entry num = 0.060733648 Billion write speed all = 0.301819541 MB/sec (with seq num) write speed all = 0.289125878 MB/sec (without seq num) 2024/11/23-19:33:08.779131 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277370: 931242 keys, 35936516 bytes, temperature: kWarm 2024/11/23-19:33:08.779214 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361588779167, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277370, "file_size": 35936516, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 29704624, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361585, "oldest_key_time": 0, "file_creation_time": 1732361585, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277370, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:08.820882 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277371.sst) /dev/shm/ToplingTemp/Topling-166939-lyaanl 2024/11/23-19:33:09.837578 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277371.sst): first pass time = 1.02's, 132.019'MB/sec 2024/11/23-19:33:11.169758 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277371.sst): old prealloc_size = 147639500, real_size = 35346384 2024/11/23-19:33:11.170367 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277371.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0334 GB avg-val = 117.46 avg-zval = 36.16 2024/11/23-19:33:11.172477 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277371.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:12.008132 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277371.sst): second pass time = 1.33's, 81.404'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.313'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2972.938'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 313.284'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34297.537'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.404'MB/sec zip pipeline throughput = 81.404'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 36.16 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 36.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5127 value = 3.2489 dict = 1.00 all = 3.3838 } Zip/UnZip{ index = 0.0951 value = 0.3078 dict = 1.00 all = 0.2955 } ---------------------------- total value len = 9.992108 GB avg = 0.162 KB (by entry num) total key len = 1.701170 GB avg = 0.028 KB total ukey len = 1.183842 GB avg = 0.016 KB total ukey num = 0.074848616 Billion total entry num = 0.061656345 Billion write speed all = 0.305468672 MB/sec (with seq num) write speed all = 0.292583264 MB/sec (without seq num) 2024/11/23-19:33:12.055334 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277371: 922697 keys, 39664970 bytes, temperature: kWarm 2024/11/23-19:33:12.055408 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361592055371, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277371, "file_size": 39664970, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33360128, "index_size": 1755392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361588, "oldest_key_time": 0, "file_creation_time": 1732361588, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277371, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:12.092357 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277372.sst) /dev/shm/ToplingTemp/Topling-166939-mOPKAm 2024/11/23-19:33:13.115265 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277372.sst): first pass time = 1.02's, 131.218'MB/sec 2024/11/23-19:33:14.427519 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277372.sst): old prealloc_size = 147639500, real_size = 32620944 2024/11/23-19:33:14.428100 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277372.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.15 avg-zval = 32.87 2024/11/23-19:33:14.429664 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277372.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:15.277357 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277372.sst): second pass time = 1.31's, 82.462'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.497'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3201.931'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 295.240'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33183.731'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.462'MB/sec zip pipeline throughput = 82.462'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 32.87 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 32.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4445 value = 3.5336 dict = 1.00 all = 3.6310 } Zip/UnZip{ index = 0.0957 value = 0.2830 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 10.106773 GB avg = 0.161 KB (by entry num) total key len = 1.727240 GB avg = 0.028 KB total ukey len = 1.202463 GB avg = 0.016 KB total ukey num = 0.075779699 Billion total entry num = 0.062587428 Billion write speed all = 0.309118766 MB/sec (with seq num) write speed all = 0.296039890 MB/sec (without seq num) 2024/11/23-19:33:15.322341 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277372: 931083 keys, 36964124 bytes, temperature: kWarm 2024/11/23-19:33:15.322412 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361595322376, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277372, "file_size": 36964124, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30605056, "index_size": 1782912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361592, "oldest_key_time": 0, "file_creation_time": 1732361592, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277372, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:15.357531 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277373.sst) /dev/shm/ToplingTemp/Topling-166939-AGQLqm 2024/11/23-19:33:16.366059 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277373.sst): first pass time = 1.01's, 133.089'MB/sec 2024/11/23-19:33:17.717007 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277373.sst): old prealloc_size = 147639500, real_size = 33314816 2024/11/23-19:33:17.717610 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277373.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926971 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.79 avg-zval = 33.77 2024/11/23-19:33:17.718268 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277373.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:18.525029 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277373.sst): second pass time = 1.35's, 80.185'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 45.523'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3101.037'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 291.835'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32676.427'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.185'MB/sec zip pipeline throughput = 80.185'MB/sec entries = 926971 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.79 avg-zval = 33.77 usrkeys = 926971 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.93 avg-zval = 33.77 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 679776 bytes = 2464208 } seq expand size = 4758432 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4434 value = 3.4580 dict = 1.00 all = 3.6489 } Zip/UnZip{ index = 0.0958 value = 0.2892 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 10.219794 GB avg = 0.161 KB (by entry num) total key len = 1.753196 GB avg = 0.028 KB total ukey len = 1.221003 GB avg = 0.016 KB total ukey num = 0.076706670 Billion total entry num = 0.063514399 Billion write speed all = 0.312722464 MB/sec (with seq num) write speed all = 0.299451005 MB/sec (without seq num) 2024/11/23-19:33:18.571613 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277373: 926971 keys, 36782858 bytes, temperature: kWarm 2024/11/23-19:33:18.571681 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361598571645, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277373, "file_size": 36782858, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 31307632, "index_size": 1775232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955188, "raw_average_key_size": 28, "raw_value_size": 108262574, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926971, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361595, "oldest_key_time": 0, "file_creation_time": 1732361595, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277373, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:18.608472 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277374.sst) /dev/shm/ToplingTemp/Topling-166939-vYMDmn 2024/11/23-19:33:19.623737 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277374.sst): first pass time = 1.02's, 132.204'MB/sec 2024/11/23-19:33:21.146844 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277374.sst): old prealloc_size = 147639500, real_size = 36833552 2024/11/23-19:33:21.147456 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277374.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926667 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.84 avg-zval = 37.58 2024/11/23-19:33:21.147761 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277374.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:21.860485 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277374.sst): second pass time = 1.52's, 71.126'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.697'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2999.816'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.392'MB/sec write SST data time = 0.12's, 312.129'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33061.113'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.126'MB/sec zip pipeline throughput = 71.126'MB/sec entries = 926667 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.84 avg-zval = 37.58 usrkeys = 926667 avg-key = 20.00 avg-zkey = 1.91 avg-val = 118.01 avg-zval = 37.58 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4448 value = 3.1088 dict = 1.00 all = 3.5026 } Zip/UnZip{ index = 0.0957 value = 0.3217 dict = 1.00 all = 0.2855 } ---------------------------- total value len = 10.329146 GB avg = 0.160 KB (by entry num) total key len = 1.779142 GB avg = 0.028 KB total ukey len = 1.239536 GB avg = 0.016 KB total ukey num = 0.077633337 Billion total entry num = 0.064441066 Billion write speed all = 0.316228778 MB/sec (with seq num) write speed all = 0.302764863 MB/sec (without seq num) 2024/11/23-19:33:21.907496 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277374: 926667 keys, 38319137 bytes, temperature: kWarm 2024/11/23-19:33:21.907564 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361601907529, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277374, "file_size": 38319137, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34827328, "index_size": 1774400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946676, "raw_average_key_size": 28, "raw_value_size": 108271134, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926667, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361598, "oldest_key_time": 0, "file_creation_time": 1732361598, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277374, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:21.945698 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277375.sst) /dev/shm/ToplingTemp/Topling-166939-59Q7mj 2024/11/23-19:33:22.934838 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277375.sst): first pass time = 0.99's, 135.696'MB/sec 2024/11/23-19:33:24.288876 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277375.sst): old prealloc_size = 147639500, real_size = 35136976 2024/11/23-19:33:24.289464 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277375.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925627 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 117.00 avg-zval = 35.81 2024/11/23-19:33:24.290072 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277375.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:25.085041 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277375.sst): second pass time = 1.35's, 80.061'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 48.224'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3115.927'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.346'MB/sec write SST data time = 0.13's, 304.629'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31138.205'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.061'MB/sec zip pipeline throughput = 80.061'MB/sec entries = 925627 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.00 avg-zval = 35.81 usrkeys = 925627 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.87 avg-zval = 35.81 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 643498 bytes = 2252256 } seq expand size = 4504486 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5135 value = 3.2675 dict = 1.00 all = 3.4959 } Zip/UnZip{ index = 0.0951 value = 0.3060 dict = 1.00 all = 0.2860 } ---------------------------- total value len = 10.441950 GB avg = 0.160 KB (by entry num) total key len = 1.805060 GB avg = 0.028 KB total ukey len = 1.258049 GB avg = 0.016 KB total ukey num = 0.078558964 Billion total entry num = 0.065366693 Billion write speed all = 0.319824816 MB/sec (with seq num) write speed all = 0.306168656 MB/sec (without seq num) 2024/11/23-19:33:25.130919 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277375: 925627 keys, 38392638 bytes, temperature: kWarm 2024/11/23-19:33:25.131003 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361605130956, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277375, "file_size": 38392638, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4217721957, "table_properties": {"data_size": 33144576, "index_size": 1760832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25917556, "raw_average_key_size": 28, "raw_value_size": 108300266, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925627, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361601, "oldest_key_time": 0, "file_creation_time": 1732361601, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277375, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:25.168052 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277376.sst) /dev/shm/ToplingTemp/Topling-166939-CbTqIj 2024/11/23-19:33:26.222350 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277376.sst): first pass time = 1.05's, 127.310'MB/sec 2024/11/23-19:33:27.572665 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277376.sst): old prealloc_size = 147639500, real_size = 35584416 2024/11/23-19:33:27.573276 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277376.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931072 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0336 GB avg-val = 116.15 avg-zval = 36.05 2024/11/23-19:33:27.575243 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277376.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:28.409555 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277376.sst): second pass time = 1.35's, 80.135'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.293'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3025.200'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 310.568'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33147.336'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.135'MB/sec zip pipeline throughput = 80.135'MB/sec entries = 931072 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 36.05 usrkeys = 931072 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 36.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931072 bytes = 3142384 } seq expand size = 6517504 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4455 value = 3.2217 dict = 1.00 all = 3.3912 } Zip/UnZip{ index = 0.0957 value = 0.3104 dict = 1.00 all = 0.2949 } ---------------------------- total value len = 10.556616 GB avg = 0.159 KB (by entry num) total key len = 1.831130 GB avg = 0.028 KB total ukey len = 1.276670 GB avg = 0.016 KB total ukey num = 0.079490036 Billion total entry num = 0.066297765 Billion write speed all = 0.323471969 MB/sec (with seq num) write speed all = 0.309622496 MB/sec (without seq num) 2024/11/23-19:33:28.457859 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277376: 931072 keys, 39578108 bytes, temperature: kWarm 2024/11/23-19:33:28.457929 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361608457893, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277376, "file_size": 39578108, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4217722267, "table_properties": {"data_size": 33568720, "index_size": 1782720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070016, "raw_average_key_size": 28, "raw_value_size": 108147832, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931072, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361605, "oldest_key_time": 0, "file_creation_time": 1732361605, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277376, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:28.494003 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277377.sst) /dev/shm/ToplingTemp/Topling-166939-KdOAjn 2024/11/23-19:33:29.448814 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277377.sst): first pass time = 0.95's, 140.575'MB/sec 2024/11/23-19:33:30.774933 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277377.sst): old prealloc_size = 147639500, real_size = 33642496 2024/11/23-19:33:30.775517 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277377.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926886 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.81 avg-zval = 34.13 2024/11/23-19:33:30.776294 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277377.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:31.625140 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277377.sst): second pass time = 1.33's, 81.688'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.256'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3160.266'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 298.394'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 38161.291'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.688'MB/sec zip pipeline throughput = 81.688'MB/sec entries = 926886 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.81 avg-zval = 34.13 usrkeys = 926886 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.81 avg-zval = 34.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926886 bytes = 3128256 } seq expand size = 6488202 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4424 value = 3.4223 dict = 1.00 all = 3.5672 } Zip/UnZip{ index = 0.0958 value = 0.2922 dict = 1.00 all = 0.2803 } ---------------------------- total value len = 10.671369 GB avg = 0.159 KB (by entry num) total key len = 1.857083 GB avg = 0.028 KB total ukey len = 1.295208 GB avg = 0.016 KB total ukey num = 0.080416922 Billion total entry num = 0.067224651 Billion write speed all = 0.327118654 MB/sec (with seq num) write speed all = 0.313076735 MB/sec (without seq num) 2024/11/23-19:33:31.670880 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277377: 926886 keys, 37625220 bytes, temperature: kWarm 2024/11/23-19:33:31.670953 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361611670914, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277377, "file_size": 37625220, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4116739076, "largest_seqno": 4217722223, "table_properties": {"data_size": 31635376, "index_size": 1775232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952808, "raw_average_key_size": 28, "raw_value_size": 108265036, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926886, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361608, "oldest_key_time": 0, "file_creation_time": 1732361608, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277377, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:31.712668 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277378.sst) /dev/shm/ToplingTemp/Topling-166939-VeQBTl 2024/11/23-19:33:32.672994 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277378.sst): first pass time = 0.96's, 139.769'MB/sec 2024/11/23-19:33:33.984331 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277378.sst): old prealloc_size = 147639500, real_size = 33318848 2024/11/23-19:33:33.984870 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277378.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923132 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0313 GB avg-val = 117.39 avg-zval = 33.94 2024/11/23-19:33:33.985747 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277378.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:34.829547 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277378.sst): second pass time = 1.31's, 82.689'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.123'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3408.455'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 295.829'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36213.896'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.689'MB/sec zip pipeline throughput = 82.689'MB/sec entries = 923132 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 33.94 usrkeys = 923132 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923132 bytes = 3115584 } seq expand size = 6461924 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5142 value = 3.4588 dict = 1.00 all = 3.5991 } Zip/UnZip{ index = 0.0951 value = 0.2891 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 10.786201 GB avg = 0.158 KB (by entry num) total key len = 1.882930 GB avg = 0.028 KB total ukey len = 1.313670 GB avg = 0.016 KB total ukey num = 0.081340054 Billion total entry num = 0.068147783 Billion write speed all = 0.330764137 MB/sec (with seq num) write speed all = 0.316530585 MB/sec (without seq num) 2024/11/23-19:33:34.875927 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277378: 923132 keys, 37292030 bytes, temperature: kWarm 2024/11/23-19:33:34.876012 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361614875960, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277378, "file_size": 37292030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4096159344, "largest_seqno": 4217721540, "table_properties": {"data_size": 31331888, "index_size": 1755968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847696, "raw_average_key_size": 28, "raw_value_size": 108370152, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923132, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361611, "oldest_key_time": 0, "file_creation_time": 1732361611, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277378, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:34.913279 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277379.sst) /dev/shm/ToplingTemp/Topling-166939-Qtp90m 2024/11/23-19:33:35.930196 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277379.sst): first pass time = 1.02's, 131.990'MB/sec 2024/11/23-19:33:37.264816 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277379.sst): old prealloc_size = 147639500, real_size = 34435040 2024/11/23-19:33:37.265384 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277379.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928853 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.50 avg-zval = 34.91 2024/11/23-19:33:37.266281 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277379.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:38.113836 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277379.sst): second pass time = 1.33's, 81.160'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.261'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3238.456'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 305.835'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32745.518'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.160'MB/sec zip pipeline throughput = 81.160'MB/sec entries = 928853 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 34.91 usrkeys = 928853 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 34.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928853 bytes = 3134896 } seq expand size = 6501971 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4450 value = 3.3373 dict = 1.00 all = 3.4928 } Zip/UnZip{ index = 0.0957 value = 0.2996 dict = 1.00 all = 0.2863 } ---------------------------- total value len = 10.900913 GB avg = 0.158 KB (by entry num) total key len = 1.908938 GB avg = 0.028 KB total ukey len = 1.332247 GB avg = 0.016 KB total ukey num = 0.082268907 Billion total entry num = 0.069076636 Billion write speed all = 0.334409356 MB/sec (with seq num) write speed all = 0.319983038 MB/sec (without seq num) 2024/11/23-19:33:38.159876 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277379: 928853 keys, 38426544 bytes, temperature: kWarm 2024/11/23-19:33:38.159948 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361618159910, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277379, "file_size": 38426544, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4137892167, "largest_seqno": 4217722023, "table_properties": {"data_size": 32424080, "index_size": 1778560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007884, "raw_average_key_size": 28, "raw_value_size": 108209846, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928853, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361614, "oldest_key_time": 0, "file_creation_time": 1732361614, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277379, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:38.197432 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277380.sst) /dev/shm/ToplingTemp/Topling-166939-XTGedl 2024/11/23-19:33:39.039917 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277380.sst): first pass time = 0.84's, 159.319'MB/sec 2024/11/23-19:33:40.335207 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277380.sst): old prealloc_size = 147639500, real_size = 31782384 2024/11/23-19:33:40.335744 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277380.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 937759 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0298 GB avg-val = 115.13 avg-zval = 31.74 2024/11/23-19:33:40.336509 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277380.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:33:41.199428 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277380.sst): second pass time = 1.29's, 83.432'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.420'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3470.381'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.318'MB/sec write SST data time = 0.13's, 282.478'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 33956.286'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.432'MB/sec zip pipeline throughput = 83.432'MB/sec entries = 937759 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.13 avg-zval = 31.74 usrkeys = 937759 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.13 avg-zval = 31.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 937759 bytes = 3164944 } seq expand size = 6564313 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5153 value = 3.6272 dict = 1.00 all = 3.7496 } Zip/UnZip{ index = 0.0951 value = 0.2757 dict = 1.00 all = 0.2667 } ---------------------------- total value len = 11.015438 GB avg = 0.157 KB (by entry num) total key len = 1.935196 GB avg = 0.028 KB total ukey len = 1.351003 GB avg = 0.016 KB total ukey num = 0.083206666 Billion total entry num = 0.070014395 Billion write speed all = 0.338057331 MB/sec (with seq num) write speed all = 0.323436344 MB/sec (without seq num) 2024/11/23-19:33:41.243127 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277380: 937759 keys, 35795418 bytes, temperature: kWarm 2024/11/23-19:33:41.243202 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361621243166, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277380, "file_size": 35795418, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4145054416, "largest_seqno": 4217722065, "table_properties": {"data_size": 29764128, "index_size": 1783616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26257252, "raw_average_key_size": 28, "raw_value_size": 107960506, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 937759, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361618, "oldest_key_time": 0, "file_creation_time": 1732361618, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277380, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:41.278130 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277381.sst) /dev/shm/ToplingTemp/Topling-166939-ASjOQl 2024/11/23-19:33:41.425646 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277381.sst): first pass time = 0.15's, 240.889'MB/sec 2024/11/23-19:33:41.768464 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277381.sst): old prealloc_size = 147639500, real_size = 9050128 2024/11/23-19:33:41.768660 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277381.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 250173 prefix = 4 raw-key = 0.0040 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0285 GB zip-val = 0.0085 GB avg-val = 114.00 avg-zval = 33.99 2024/11/23-19:33:41.768886 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277381.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.036 GB 2024/11/23-19:33:41.993356 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00/277381.sst): second pass time = 0.34's, 83.354'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 44.748'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2776.808'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.329'MB/sec write SST data time = 0.03's, 290.690'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.291'MB, throughput = 9652.113'MB/sec zip my value time = 0.34's, unzip length = 0.029'GB zip my value throughput = 83.354'MB/sec zip pipeline throughput = 83.354'MB/sec entries = 250173 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.00 avg-zval = 33.99 usrkeys = 250173 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.00 avg-zval = 33.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 250173 bytes = 781808 } seq expand size = 1751211 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0050 GB value = 0.0285 GB dict = 0.29 MB all = 0.0355 GB } __ZipSize{ index = 0.0005 GB value = 0.0085 GB dict = 0.29 MB all = 0.0101 GB } UnZip/Zip{ index = 10.3480 value = 3.3537 dict = 1.00 all = 3.5304 } Zip/UnZip{ index = 0.0966 value = 0.2982 dict = 1.00 all = 0.2833 } ---------------------------- total value len = 11.045709 GB avg = 0.157 KB (by entry num) total key len = 1.942200 GB avg = 0.028 KB total ukey len = 1.356006 GB avg = 0.016 KB total ukey num = 0.083456839 Billion total entry num = 0.070264568 Billion write speed all = 0.339023335 MB/sec (with seq num) write speed all = 0.324350409 MB/sec (without seq num) 2024/11/23-19:33:42.010271 139957248734784 [compaction_job.cc:1948] [default] [JOB 34283] Generated table #277381: 250173 keys, 10062451 bytes, temperature: kWarm 2024/11/23-19:33:42.010333 139957248734784 EVENT_LOG_v1 {"time_micros": 1732361622010300, "cf_name": "default", "job": 34283, "event": "table_file_creation", "file_number": 277381, "file_size": 10062451, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4187186584, "largest_seqno": 4212061860, "table_properties": {"data_size": 8503872, "index_size": 483520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7004844, "raw_average_key_size": 28, "raw_value_size": 28519722, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 250173, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732361621, "oldest_key_time": 0, "file_creation_time": 1732361621, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277381, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:33:42.023728 139957248734784 [compaction_job.cc:703] job-34283: subcompact[0], size: 0.735492 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34283/att-00 [277362,277363,277364,277365,277366,277367,277368,277369,277370,277371,277372,277373,277374,277375,277376,277377,277378,277379,277380,277381] 2024/11/23-19:33:42.141088 139957248734784 (Original Log Time 2024/11/23-19:32:36.141435) EVENT_LOG_v1 {"time_micros": 1732361556141385, "job": 34283, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [277333], "files_L3": [277352, 277353, 277279, 277280, 277281, 277282, 277283, 277284, 277285, 277286, 277287, 277288, 277289, 277290, 277291, 277303, 277304, 277305, 277306, 277354], "score": 0.999932, "input_data_size": 756148831, "oldest_snapshot_seqno": 4217972544} 2024/11/23-19:33:42.141091 139957248734784 (Original Log Time 2024/11/23-19:33:42.139533) [compaction_job.cc:2022] [default] [JOB 34283] Compacted 1@2 + 20@3 files to L3 => 735491764 bytes 2024/11/23-19:33:42.141093 139957248734784 (Original Log Time 2024/11/23-19:33:42.141014) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.5 rd, 11.2 wr, level 3, files in(1, 20) out(20 +0 blob) MB in(27.1, 694.0 +0.0 blob) out(701.4 +0.0 blob), read-write-amplify(52.4) write-amplify(25.8) OK, records in: 17992615, records dropped: 86211 output_compression: Snappy 2024/11/23-19:33:42.141096 139957248734784 (Original Log Time 2024/11/23-19:33:42.141059) EVENT_LOG_v1 {"time_micros": 1732361622141037, "cf": "default", "job": 34283, "event": "compaction_finished", "compaction_time_micros": 65882302, "compaction_time_cpu_micros": 27887991, "output_level": 3, "num_output_files": 20, "total_output_size": 735491764, "num_input_records": 17992615, "num_output_records": 17906404, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 539876074, "file_range_sync_nanos": 0, "file_fsync_nanos": 14930, "file_prepare_write_nanos": 59422, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-19:33:42.272601 139957248734784 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732361555,"executesMs":66128,"compactionJobId":34283,"attempt":0,"compactionInputRawBytes":2597904862,"compactionInputZipBytes":756148831,"compactionOutputRawBytes":0,"compactionOutputZipBytes":735491764,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-6575edb52cba5a925a34cb77aca6fde0-3d24b1c0ef59e77d-00"}}, response = 2024/11/23-19:33:42.272686 139957248734784 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34283/att-00: olev 3, work 66.128 s, result 7.840 ms, install 1.578 ms, input{raw 2.419 GiB zip 721.120 MiB}