2024/11/23-06:18:31.489920 140025146283584 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00: fork to child time = 0.009015 sec 2024/11/23-06:18:31.490702 140025146283584 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00 2024/11/23-06:18:31.498431 140025146283584 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00 2024/11/23-06:18:31.498966 140025146283584 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-06:18:31.499251 140025146283584 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-06:18:31.581187 140025146283584 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20901/att-00/MANIFEST-206347 2024/11/23-06:18:31.594233 140025146283584 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20901/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 248253, last_sequence is 3831769291, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 248085 2024/11/23-06:18:31.594240 140025146283584 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-06:18:31.594659 140025146283584 [version_set.cc:5721] Creating manifest 248253 2024/11/23-06:18:31.647861 140025146283584 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-06:18:31.676823 140025146283584 [compaction_job.cc:2437] [default] [JOB 20901] Compacting 1@2 + 16@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-06:18:31.676843 140025146283584 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 15637 Base level 2, inputs: [248215(26MB)], [248136(34MB) 248137(35MB) 248138(33MB) 248139(35MB) 248140(33MB) 248141(35MB) 248142(35MB) 248143(35MB) 248144(35MB) 248145(36MB) 248146(35MB) 248147(36MB) 248148(35MB) 248149(34MB) 248150(35MB) 248239(35MB)] 2024/11/23-06:18:31.682611 140025146283584 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:18:31.682642 140025146283584 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:18:31.682661 140025146283584 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:18:31.682666 140025146283584 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:18:31.682677 140025146283584 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:18:32.113561 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248254.sst) /dev/shm/ToplingTemp/Topling-432343-QARZJX 2024/11/23-06:18:32.809737 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248254.sst): first pass time = 0.70's, 192.808'MB/sec 2024/11/23-06:18:37.547164 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248254.sst): old prealloc_size = 147639500, real_size = 32091584 2024/11/23-06:18:37.547726 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248254.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927825 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0301 GB avg-val = 116.66 avg-zval = 32.42 2024/11/23-06:18:37.548484 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248254.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:18:39.012737 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248254.sst): second pass time = 4.74's, 22.852'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.47's, 24.616'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3337.816'MB/sec (index lex order gen) rebuild zvType time = 1.23's, 0.189'MB/sec write SST data time = 0.24's, 151.669'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35823.145'MB/sec zip my value time = 4.74's, unzip length = 0.108'GB zip my value throughput = 22.852'MB/sec zip pipeline throughput = 133.124'MB/sec entries = 927825 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.66 avg-zval = 32.42 usrkeys = 927825 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.66 avg-zval = 32.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927825 bytes = 3131424 } seq expand size = 6494775 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4428 value = 3.5981 dict = 1.00 all = 3.7203 } Zip/UnZip{ index = 0.0958 value = 0.2779 dict = 1.00 all = 0.2688 } ---------------------------- total value len = 0.114733 GB avg = 0.124 KB (by entry num) total key len = 0.025979 GB avg = 0.028 KB total ukey len = 0.018557 GB avg = 0.020 KB total ukey num = 0.000927825 Billion total entry num = 0.000927825 Billion write speed all = 0.001823271 MB/sec (with seq num) write speed all = 0.001727093 MB/sec (without seq num) 2024/11/23-06:18:39.063250 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248254: 927825 keys, 36076784 bytes, temperature: kWarm 2024/11/23-06:18:39.063330 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313919063281, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248254, "file_size": 36076784, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3726348054, "largest_seqno": 3831416771, "table_properties": {"data_size": 30082480, "index_size": 1776960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979100, "raw_average_key_size": 28, "raw_value_size": 108238718, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927825, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313912, "oldest_key_time": 0, "file_creation_time": 1732313912, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248254, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:18:39.100265 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248255.sst) /dev/shm/ToplingTemp/Topling-432343-kDRcBV 2024/11/23-06:18:39.975674 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248255.sst): first pass time = 0.88's, 153.326'MB/sec 2024/11/23-06:18:41.963449 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248255.sst): old prealloc_size = 147639500, real_size = 32705856 2024/11/23-06:18:41.964127 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248255.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928036 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.63 avg-zval = 33.09 2024/11/23-06:18:41.964971 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248255.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:18:42.971088 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248255.sst): second pass time = 1.99's, 54.476'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 36.645'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2687.955'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.274'MB/sec write SST data time = 0.16's, 228.676'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 38255.146'MB/sec zip my value time = 1.99's, unzip length = 0.108'GB zip my value throughput = 54.476'MB/sec zip pipeline throughput = 107.012'MB/sec entries = 928036 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.63 avg-zval = 33.09 usrkeys = 928036 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.63 avg-zval = 33.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928036 bytes = 3364144 } seq expand size = 6496252 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5141 value = 3.5245 dict = 1.00 all = 3.6350 } Zip/UnZip{ index = 0.0951 value = 0.2837 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 0.229463 GB avg = 0.124 KB (by entry num) total key len = 0.051964 GB avg = 0.028 KB total ukey len = 0.037117 GB avg = 0.020 KB total ukey num = 0.001855861 Billion total entry num = 0.001855861 Billion write speed all = 0.003646374 MB/sec (with seq num) write speed all = 0.003454006 MB/sec (without seq num) 2024/11/23-06:18:43.008857 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248255: 928036 keys, 36923620 bytes, temperature: kWarm 2024/11/23-06:18:43.008933 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313923008889, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248255, "file_size": 36923620, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3831418634, "table_properties": {"data_size": 30708336, "index_size": 1765312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25985008, "raw_average_key_size": 28, "raw_value_size": 108232820, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928036, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313919, "oldest_key_time": 0, "file_creation_time": 1732313919, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248255, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:18:43.083585 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248256.sst) /dev/shm/ToplingTemp/Topling-432343-ZPKDmX 2024/11/23-06:18:43.926769 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248256.sst): first pass time = 0.84's, 159.188'MB/sec 2024/11/23-06:18:45.485933 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248256.sst): old prealloc_size = 147639500, real_size = 30970544 2024/11/23-06:18:45.486586 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248256.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928104 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0290 GB avg-val = 116.62 avg-zval = 31.20 2024/11/23-06:18:45.487376 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248256.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:18:46.368434 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248256.sst): second pass time = 1.56's, 69.463'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 39.742'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2809.426'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.15's, 239.702'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33591.537'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.463'MB/sec zip pipeline throughput = 109.018'MB/sec entries = 928104 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.62 avg-zval = 31.20 usrkeys = 928104 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.62 avg-zval = 31.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928104 bytes = 3248384 } seq expand size = 6496728 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4418 value = 3.7372 dict = 1.00 all = 3.8269 } Zip/UnZip{ index = 0.0958 value = 0.2676 dict = 1.00 all = 0.2613 } ---------------------------- total value len = 0.344190 GB avg = 0.124 KB (by entry num) total key len = 0.077951 GB avg = 0.028 KB total ukey len = 0.055679 GB avg = 0.020 KB total ukey num = 0.002783965 Billion total entry num = 0.002783965 Billion write speed all = 0.005469335 MB/sec (with seq num) write speed all = 0.005180779 MB/sec (without seq num) 2024/11/23-06:18:46.403166 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248256: 928104 keys, 35072416 bytes, temperature: kWarm 2024/11/23-06:18:46.403225 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313926403193, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248256, "file_size": 35072416, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3684825109, "largest_seqno": 3831417806, "table_properties": {"data_size": 28960672, "index_size": 1777664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986912, "raw_average_key_size": 28, "raw_value_size": 108230884, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928104, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313923, "oldest_key_time": 0, "file_creation_time": 1732313923, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248256, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:18:46.447772 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248257.sst) /dev/shm/ToplingTemp/Topling-432343-wSYpUW 2024/11/23-06:18:47.349169 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248257.sst): first pass time = 0.90's, 148.907'MB/sec 2024/11/23-06:18:48.912326 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248257.sst): old prealloc_size = 147639500, real_size = 32624176 2024/11/23-06:18:48.912924 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248257.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928102 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.62 avg-zval = 33.00 2024/11/23-06:18:48.913722 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248257.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:18:49.830876 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248257.sst): second pass time = 1.56's, 69.270'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.110'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3058.541'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 281.642'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33616.664'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.270'MB/sec zip pipeline throughput = 114.022'MB/sec entries = 928102 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.62 avg-zval = 33.00 usrkeys = 928102 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.62 avg-zval = 33.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928102 bytes = 3364384 } seq expand size = 6496714 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5122 value = 3.5339 dict = 1.00 all = 3.6431 } Zip/UnZip{ index = 0.0951 value = 0.2830 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 0.458918 GB avg = 0.124 KB (by entry num) total key len = 0.103938 GB avg = 0.028 KB total ukey len = 0.074241 GB avg = 0.020 KB total ukey num = 0.003712067 Billion total entry num = 0.003712067 Billion write speed all = 0.007292131 MB/sec (with seq num) write speed all = 0.006907395 MB/sec (without seq num) 2024/11/23-06:18:49.865350 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248257: 928102 keys, 36842084 bytes, temperature: kWarm 2024/11/23-06:18:49.865411 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313929865380, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248257, "file_size": 36842084, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3831419571, "table_properties": {"data_size": 30626208, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986856, "raw_average_key_size": 28, "raw_value_size": 108231004, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928102, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313926, "oldest_key_time": 0, "file_creation_time": 1732313926, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248257, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:18:49.902664 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248258.sst) /dev/shm/ToplingTemp/Topling-432343-TxSMmW 2024/11/23-06:18:50.722367 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248258.sst): first pass time = 0.82's, 163.747'MB/sec 2024/11/23-06:18:52.257300 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248258.sst): old prealloc_size = 147639500, real_size = 30634368 2024/11/23-06:18:52.257900 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248258.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927825 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.66 avg-zval = 30.84 2024/11/23-06:18:52.258654 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248258.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:18:53.120635 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248258.sst): second pass time = 1.53's, 70.552'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.099'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3085.329'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 263.785'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33348.730'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.552'MB/sec zip pipeline throughput = 130.072'MB/sec entries = 927825 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.66 avg-zval = 30.84 usrkeys = 927825 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.66 avg-zval = 30.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927825 bytes = 3131424 } seq expand size = 6494775 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.3793 value = 3.7827 dict = 1.00 all = 3.8771 } Zip/UnZip{ index = 0.0963 value = 0.2644 dict = 1.00 all = 0.2579 } ---------------------------- total value len = 0.573651 GB avg = 0.124 KB (by entry num) total key len = 0.129917 GB avg = 0.028 KB total ukey len = 0.092798 GB avg = 0.020 KB total ukey num = 0.004639892 Billion total entry num = 0.004639892 Billion write speed all = 0.009114757 MB/sec (with seq num) write speed all = 0.008633877 MB/sec (without seq num) 2024/11/23-06:18:53.152498 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248258: 927825 keys, 34617940 bytes, temperature: kWarm 2024/11/23-06:18:53.152564 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313933152529, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248258, "file_size": 34617940, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3700405246, "largest_seqno": 3831414027, "table_properties": {"data_size": 28614384, "index_size": 1787840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979100, "raw_average_key_size": 28, "raw_value_size": 108238706, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927825, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313929, "oldest_key_time": 0, "file_creation_time": 1732313929, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248258, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:18:53.188017 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248259.sst) /dev/shm/ToplingTemp/Topling-432343-NhvXfX 2024/11/23-06:18:54.055338 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248259.sst): first pass time = 0.87's, 154.757'MB/sec 2024/11/23-06:18:55.668041 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248259.sst): old prealloc_size = 147639500, real_size = 32922208 2024/11/23-06:18:55.668597 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248259.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927994 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.63 avg-zval = 33.31 2024/11/23-06:18:55.669402 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248259.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:18:56.592871 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248259.sst): second pass time = 1.61's, 67.145'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.159'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3327.974'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 282.532'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35500.904'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.145'MB/sec zip pipeline throughput = 134.282'MB/sec entries = 927994 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.63 avg-zval = 33.31 usrkeys = 927994 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.63 avg-zval = 33.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927994 bytes = 3364000 } seq expand size = 6495958 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4455 value = 3.5012 dict = 1.00 all = 3.6139 } Zip/UnZip{ index = 0.0957 value = 0.2856 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 0.688381 GB avg = 0.124 KB (by entry num) total key len = 0.155901 GB avg = 0.028 KB total ukey len = 0.111358 GB avg = 0.020 KB total ukey num = 0.005567886 Billion total entry num = 0.005567886 Billion write speed all = 0.010937218 MB/sec (with seq num) write speed all = 0.010360186 MB/sec (without seq num) 2024/11/23-06:18:56.641915 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248259: 927994 keys, 37139744 bytes, temperature: kWarm 2024/11/23-06:18:56.641980 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313936641947, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248259, "file_size": 37139744, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3831417580, "table_properties": {"data_size": 30913168, "index_size": 1776832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25983832, "raw_average_key_size": 28, "raw_value_size": 108233960, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927994, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313933, "oldest_key_time": 0, "file_creation_time": 1732313933, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248259, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:18:56.678859 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248260.sst) /dev/shm/ToplingTemp/Topling-432343-fcxIcX 2024/11/23-06:18:57.538255 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248260.sst): first pass time = 0.86's, 156.183'MB/sec 2024/11/23-06:18:59.050292 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248260.sst): old prealloc_size = 147639500, real_size = 33348992 2024/11/23-06:18:59.050887 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248260.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921594 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0314 GB avg-val = 117.64 avg-zval = 34.03 2024/11/23-06:18:59.051653 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248260.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:18:59.953445 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248260.sst): second pass time = 1.51's, 71.737'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 41.448'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3113.867'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.298'MB/sec write SST data time = 0.13's, 292.151'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30498.621'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.737'MB/sec zip pipeline throughput = 103.498'MB/sec entries = 921594 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.64 avg-zval = 34.03 usrkeys = 921594 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.64 avg-zval = 34.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921594 bytes = 3225600 } seq expand size = 6451158 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5124 value = 3.4565 dict = 1.00 all = 3.5855 } Zip/UnZip{ index = 0.0951 value = 0.2893 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 0.803246 GB avg = 0.124 KB (by entry num) total key len = 0.181705 GB avg = 0.028 KB total ukey len = 0.129790 GB avg = 0.020 KB total ukey num = 0.006489480 Billion total entry num = 0.006489480 Billion write speed all = 0.012758953 MB/sec (with seq num) write speed all = 0.012086441 MB/sec (without seq num) 2024/11/23-06:18:59.987210 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248260: 921594 keys, 37433526 bytes, temperature: kWarm 2024/11/23-06:18:59.987274 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313939987241, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248260, "file_size": 37433526, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3690138834, "largest_seqno": 3831419351, "table_properties": {"data_size": 31365040, "index_size": 1753344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25804632, "raw_average_key_size": 28, "raw_value_size": 108413188, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921594, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313936, "oldest_key_time": 0, "file_creation_time": 1732313936, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248260, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:00.025947 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248261.sst) /dev/shm/ToplingTemp/Topling-432343-1iutaV 2024/11/23-06:19:00.923291 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248261.sst): first pass time = 0.90's, 149.579'MB/sec 2024/11/23-06:19:02.390200 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248261.sst): old prealloc_size = 147639500, real_size = 33067152 2024/11/23-06:19:02.390801 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248261.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928286 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.59 avg-zval = 33.46 2024/11/23-06:19:02.391599 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248261.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:19:03.353900 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248261.sst): second pass time = 1.47's, 73.815'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.690'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3047.823'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.13's, 285.460'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 25825.656'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.815'MB/sec zip pipeline throughput = 74.441'MB/sec entries = 928286 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.59 avg-zval = 33.46 usrkeys = 928286 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.59 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928286 bytes = 3365056 } seq expand size = 6498002 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4476 value = 3.4847 dict = 1.00 all = 3.5998 } Zip/UnZip{ index = 0.0957 value = 0.2870 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 0.917969 GB avg = 0.124 KB (by entry num) total key len = 0.207697 GB avg = 0.028 KB total ukey len = 0.148355 GB avg = 0.020 KB total ukey num = 0.007417766 Billion total entry num = 0.007417766 Billion write speed all = 0.014581129 MB/sec (with seq num) write speed all = 0.013812451 MB/sec (without seq num) 2024/11/23-06:19:03.388622 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248261: 928286 keys, 37285248 bytes, temperature: kWarm 2024/11/23-06:19:03.388688 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313943388654, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248261, "file_size": 37285248, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3831419624, "table_properties": {"data_size": 31057856, "index_size": 1777024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25992008, "raw_average_key_size": 28, "raw_value_size": 108225828, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928286, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313940, "oldest_key_time": 0, "file_creation_time": 1732313940, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248261, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:03.426960 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248262.sst) /dev/shm/ToplingTemp/Topling-432343-4MOH6Y 2024/11/23-06:19:04.337686 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248262.sst): first pass time = 0.91's, 147.381'MB/sec 2024/11/23-06:19:05.965529 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248262.sst): old prealloc_size = 147639500, real_size = 33092608 2024/11/23-06:19:05.966060 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248262.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928005 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.63 avg-zval = 33.51 2024/11/23-06:19:05.966832 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248262.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:19:06.914327 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248262.sst): second pass time = 1.63's, 66.515'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.080'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3441.241'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.13's, 280.372'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32327.635'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.515'MB/sec zip pipeline throughput = 132.988'MB/sec entries = 928005 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.63 avg-zval = 33.51 usrkeys = 928005 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.63 avg-zval = 33.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928005 bytes = 3132032 } seq expand size = 6496035 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5119 value = 3.4808 dict = 1.00 all = 3.6198 } Zip/UnZip{ index = 0.0951 value = 0.2873 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 1.032699 GB avg = 0.124 KB (by entry num) total key len = 0.233682 GB avg = 0.028 KB total ukey len = 0.166915 GB avg = 0.020 KB total ukey num = 0.008345771 Billion total entry num = 0.008345771 Billion write speed all = 0.016403085 MB/sec (with seq num) write speed all = 0.015538281 MB/sec (without seq num) 2024/11/23-06:19:06.952003 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248262: 928005 keys, 37078320 bytes, temperature: kWarm 2024/11/23-06:19:06.952058 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313946952028, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248262, "file_size": 37078320, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3753310698, "largest_seqno": 3831417471, "table_properties": {"data_size": 31094768, "index_size": 1765632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984140, "raw_average_key_size": 28, "raw_value_size": 108233722, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928005, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313943, "oldest_key_time": 0, "file_creation_time": 1732313943, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248262, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:06.989932 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248263.sst) /dev/shm/ToplingTemp/Topling-432343-1XufkV 2024/11/23-06:19:07.881570 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248263.sst): first pass time = 0.89's, 150.536'MB/sec 2024/11/23-06:19:09.436661 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248263.sst): old prealloc_size = 147639500, real_size = 35437280 2024/11/23-06:19:09.437238 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248263.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932874 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0334 GB avg-val = 115.88 avg-zval = 35.81 2024/11/23-06:19:09.437661 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248263.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:19:10.196758 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248263.sst): second pass time = 1.55's, 69.540'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 49.843'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3236.308'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.370'MB/sec write SST data time = 0.13's, 292.044'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33543.638'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.540'MB/sec zip pipeline throughput = 69.540'MB/sec entries = 932874 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.88 avg-zval = 35.81 usrkeys = 932874 avg-key = 20.00 avg-zkey = 1.93 avg-val = 118.77 avg-zval = 35.81 TagRS{ kinds = RS_Key0_TagN bytes = 145880 } TagArray{ size = 386054 bytes = 1447712 } seq expand size = 2702378 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.3763 value = 3.2359 dict = 1.00 all = 3.5428 } Zip/UnZip{ index = 0.0964 value = 0.3090 dict = 1.00 all = 0.2823 } ---------------------------- total value len = 1.143499 GB avg = 0.123 KB (by entry num) total key len = 0.259802 GB avg = 0.028 KB total ukey len = 0.185573 GB avg = 0.020 KB total ukey num = 0.009278645 Billion total entry num = 0.009278645 Billion write speed all = 0.018175801 MB/sec (with seq num) write speed all = 0.017214372 MB/sec (without seq num) 2024/11/23-06:19:10.236876 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248263: 932874 keys, 37884937 bytes, temperature: kWarm 2024/11/23-06:19:10.236938 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313950236905, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248263, "file_size": 37884937, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3831416288, "table_properties": {"data_size": 33405776, "index_size": 1798080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26120472, "raw_average_key_size": 28, "raw_value_size": 108097260, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932874, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313946, "oldest_key_time": 0, "file_creation_time": 1732313946, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248263, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:10.275880 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248264.sst) /dev/shm/ToplingTemp/Topling-432343-r2nifV 2024/11/23-06:19:11.102835 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248264.sst): first pass time = 0.83's, 162.310'MB/sec 2024/11/23-06:19:12.764367 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248264.sst): old prealloc_size = 147639500, real_size = 36393472 2024/11/23-06:19:12.764940 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248264.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926827 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.81 avg-zval = 37.11 2024/11/23-06:19:12.764987 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248264.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:19:13.406855 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248264.sst): second pass time = 1.66's, 65.189'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 57.982'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3186.699'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.448'MB/sec write SST data time = 0.13's, 297.390'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34363.237'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.189'MB/sec zip pipeline throughput = 65.189'MB/sec entries = 926827 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 37.11 usrkeys = 926827 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 37.11 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5122 value = 3.1474 dict = 1.00 all = 3.6030 } Zip/UnZip{ index = 0.0951 value = 0.3177 dict = 1.00 all = 0.2775 } ---------------------------- total value len = 1.251766 GB avg = 0.123 KB (by entry num) total key len = 0.285753 GB avg = 0.028 KB total ukey len = 0.204109 GB avg = 0.020 KB total ukey num = 0.010205472 Billion total entry num = 0.010205472 Billion write speed all = 0.019913386 MB/sec (with seq num) write speed all = 0.018855966 MB/sec (without seq num) 2024/11/23-06:19:13.441120 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248264: 926827 keys, 37251376 bytes, temperature: kWarm 2024/11/23-06:19:13.441191 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313953441159, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248264, "file_size": 37251376, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34398256, "index_size": 1763328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951156, "raw_average_key_size": 28, "raw_value_size": 108266662, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926827, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313950, "oldest_key_time": 0, "file_creation_time": 1732313950, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248264, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:13.478669 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248265.sst) /dev/shm/ToplingTemp/Topling-432343-xV1kLY 2024/11/23-06:19:14.392414 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248265.sst): first pass time = 0.91's, 146.895'MB/sec 2024/11/23-06:19:15.735659 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248265.sst): old prealloc_size = 147639500, real_size = 34954624 2024/11/23-06:19:15.736220 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248265.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925733 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0330 GB avg-val = 116.99 avg-zval = 35.61 2024/11/23-06:19:15.736939 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248265.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:19:16.619628 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248265.sst): second pass time = 1.34's, 80.664'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 43.945'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3259.464'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 292.600'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 24263.520'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.664'MB/sec zip pipeline throughput = 80.664'MB/sec entries = 925733 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.99 avg-zval = 35.61 usrkeys = 925733 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.81 avg-zval = 35.61 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 769889 bytes = 2887104 } seq expand size = 5389223 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5113 value = 3.2856 dict = 1.00 all = 3.4552 } Zip/UnZip{ index = 0.0951 value = 0.3044 dict = 1.00 all = 0.2894 } ---------------------------- total value len = 1.365452 GB avg = 0.123 KB (by entry num) total key len = 0.311674 GB avg = 0.028 KB total ukey len = 0.222624 GB avg = 0.020 KB total ukey num = 0.011131205 Billion total entry num = 0.011131205 Billion write speed all = 0.021720622 MB/sec (with seq num) write speed all = 0.020567331 MB/sec (without seq num) 2024/11/23-06:19:16.657515 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248265: 925733 keys, 38845030 bytes, temperature: kWarm 2024/11/23-06:19:16.657580 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313956657546, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248265, "file_size": 38845030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3831419312, "table_properties": {"data_size": 32961584, "index_size": 1761408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920524, "raw_average_key_size": 28, "raw_value_size": 108297246, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925733, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313953, "oldest_key_time": 0, "file_creation_time": 1732313953, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248265, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:16.700175 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248266.sst) /dev/shm/ToplingTemp/Topling-432343-MciaaZ 2024/11/23-06:19:17.495567 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248266.sst): first pass time = 0.80's, 168.753'MB/sec 2024/11/23-06:19:18.853353 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248266.sst): old prealloc_size = 147639500, real_size = 33194080 2024/11/23-06:19:18.853908 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248266.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928256 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.59 avg-zval = 33.61 2024/11/23-06:19:18.854683 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248266.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:19:19.799372 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248266.sst): second pass time = 1.36's, 79.746'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.425'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3314.743'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.13's, 289.441'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33982.628'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.746'MB/sec zip pipeline throughput = 79.746'MB/sec entries = 928256 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.59 avg-zval = 33.61 usrkeys = 928256 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.59 avg-zval = 33.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928256 bytes = 3248912 } seq expand size = 6497792 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5113 value = 3.4693 dict = 1.00 all = 3.5987 } Zip/UnZip{ index = 0.0951 value = 0.2882 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 1.480176 GB avg = 0.123 KB (by entry num) total key len = 0.337665 GB avg = 0.028 KB total ukey len = 0.241189 GB avg = 0.020 KB total ukey num = 0.012059461 Billion total entry num = 0.012059461 Billion write speed all = 0.023542074 MB/sec (with seq num) write speed all = 0.022292659 MB/sec (without seq num) 2024/11/23-06:19:19.834813 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248266: 928256 keys, 37296552 bytes, temperature: kWarm 2024/11/23-06:19:19.834880 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313959834847, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248266, "file_size": 37296552, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3831418382, "table_properties": {"data_size": 31195600, "index_size": 1766208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25991168, "raw_average_key_size": 28, "raw_value_size": 108226644, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928256, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313956, "oldest_key_time": 0, "file_creation_time": 1732313956, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248266, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:19.872129 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248267.sst) /dev/shm/ToplingTemp/Topling-432343-gARbCX 2024/11/23-06:19:20.659444 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248267.sst): first pass time = 0.79's, 170.483'MB/sec 2024/11/23-06:19:21.960052 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248267.sst): old prealloc_size = 147639500, real_size = 30536608 2024/11/23-06:19:21.960588 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248267.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928351 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0285 GB avg-val = 116.58 avg-zval = 30.73 2024/11/23-06:19:21.961360 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248267.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:19:22.819717 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248267.sst): second pass time = 1.30's, 83.251'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.159'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3460.082'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 266.417'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32492.266'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.251'MB/sec zip pipeline throughput = 83.251'MB/sec entries = 928351 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.58 avg-zval = 30.73 usrkeys = 928351 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.58 avg-zval = 30.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928351 bytes = 3133200 } seq expand size = 6498457 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4442 value = 3.7938 dict = 1.00 all = 3.8878 } Zip/UnZip{ index = 0.0957 value = 0.2636 dict = 1.00 all = 0.2572 } ---------------------------- total value len = 1.594899 GB avg = 0.123 KB (by entry num) total key len = 0.363659 GB avg = 0.028 KB total ukey len = 0.259756 GB avg = 0.020 KB total ukey num = 0.012987812 Billion total entry num = 0.012987812 Billion write speed all = 0.025363438 MB/sec (with seq num) write speed all = 0.024017894 MB/sec (without seq num) 2024/11/23-06:19:22.850238 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248267: 928351 keys, 34522870 bytes, temperature: kWarm 2024/11/23-06:19:22.850301 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313962850269, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248267, "file_size": 34522870, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3711254101, "largest_seqno": 3831418024, "table_properties": {"data_size": 28526608, "index_size": 1777728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25993828, "raw_average_key_size": 28, "raw_value_size": 108224014, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928351, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313959, "oldest_key_time": 0, "file_creation_time": 1732313959, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248267, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:22.885681 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248268.sst) /dev/shm/ToplingTemp/Topling-432343-06NSsX 2024/11/23-06:19:23.759406 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248268.sst): first pass time = 0.87's, 153.622'MB/sec 2024/11/23-06:19:25.091880 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248268.sst): old prealloc_size = 147639500, real_size = 31842864 2024/11/23-06:19:25.092430 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248268.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928654 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0298 GB avg-val = 116.53 avg-zval = 32.12 2024/11/23-06:19:25.093213 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248268.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:19:25.986232 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248268.sst): second pass time = 1.33's, 81.257'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.194'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3357.995'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.303'MB/sec write SST data time = 0.13's, 280.494'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33512.135'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.257'MB/sec zip pipeline throughput = 81.257'MB/sec entries = 928654 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.53 avg-zval = 32.12 usrkeys = 928654 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.53 avg-zval = 32.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928654 bytes = 3250304 } seq expand size = 6500578 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.4454 value = 3.6274 dict = 1.00 all = 3.7338 } Zip/UnZip{ index = 0.0957 value = 0.2757 dict = 1.00 all = 0.2678 } ---------------------------- total value len = 1.709615 GB avg = 0.123 KB (by entry num) total key len = 0.389661 GB avg = 0.028 KB total ukey len = 0.278329 GB avg = 0.020 KB total ukey num = 0.013916466 Billion total entry num = 0.013916466 Billion write speed all = 0.027184633 MB/sec (with seq num) write speed all = 0.025742940 MB/sec (without seq num) 2024/11/23-06:19:26.021724 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248268: 928654 keys, 35946724 bytes, temperature: kWarm 2024/11/23-06:19:26.021794 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313966021753, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248268, "file_size": 35946724, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3831419059, "table_properties": {"data_size": 29832416, "index_size": 1778112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002312, "raw_average_key_size": 28, "raw_value_size": 108215420, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928654, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313962, "oldest_key_time": 0, "file_creation_time": 1732313962, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248268, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:26.059439 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248269.sst) /dev/shm/ToplingTemp/Topling-432343-b1YHJW 2024/11/23-06:19:26.974926 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248269.sst): first pass time = 0.92's, 146.614'MB/sec 2024/11/23-06:19:28.302327 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248269.sst): old prealloc_size = 147639500, real_size = 32657728 2024/11/23-06:19:28.302881 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248269.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928664 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.53 avg-zval = 33.00 2024/11/23-06:19:28.303659 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248269.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:19:29.219992 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248269.sst): second pass time = 1.33's, 81.568'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.060'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3342.179'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 288.657'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33241.065'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.568'MB/sec zip pipeline throughput = 81.568'MB/sec entries = 928664 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.53 avg-zval = 33.00 usrkeys = 928664 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.53 avg-zval = 33.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928664 bytes = 3250336 } seq expand size = 6500648 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4448 value = 3.5310 dict = 1.00 all = 3.6510 } Zip/UnZip{ index = 0.0957 value = 0.2832 dict = 1.00 all = 0.2739 } ---------------------------- total value len = 1.824331 GB avg = 0.123 KB (by entry num) total key len = 0.415664 GB avg = 0.028 KB total ukey len = 0.296903 GB avg = 0.020 KB total ukey num = 0.014845130 Billion total entry num = 0.014845130 Billion write speed all = 0.029005656 MB/sec (with seq num) write speed all = 0.027467821 MB/sec (without seq num) 2024/11/23-06:19:29.254449 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248269: 928664 keys, 36761516 bytes, temperature: kWarm 2024/11/23-06:19:29.254512 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313969254479, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248269, "file_size": 36761516, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3831419147, "table_properties": {"data_size": 30647152, "index_size": 1778240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002592, "raw_average_key_size": 28, "raw_value_size": 108215228, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928664, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313966, "oldest_key_time": 0, "file_creation_time": 1732313966, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248269, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:29.293405 140025146283584 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248270.sst) /dev/shm/ToplingTemp/Topling-432343-t4ggDY 2024/11/23-06:19:29.375413 140025146283584 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248270.sst): first pass time = 0.08's, 234.621'MB/sec 2024/11/23-06:19:29.543395 140025146283584 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248270.sst): old prealloc_size = 147639500, real_size = 5657168 2024/11/23-06:19:29.543520 140025146283584 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248270.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 132447 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0155 GB zip-val = 0.0054 GB avg-val = 117.21 avg-zval = 40.54 2024/11/23-06:19:29.543667 140025146283584 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248270.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.019 GB 2024/11/23-06:19:29.666879 140025146283584 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00/248270.sst): second pass time = 0.17's, 92.735'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 50.361'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2509.396'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.317'MB/sec write SST data time = 0.02's, 330.314'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.162'MB, throughput = 5691.060'MB/sec zip my value time = 0.17's, unzip length = 0.016'GB zip my value throughput = 92.735'MB/sec zip pipeline throughput = 92.735'MB/sec entries = 132447 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.21 avg-zval = 40.54 usrkeys = 132447 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.21 avg-zval = 40.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 132447 bytes = 430464 } seq expand size = 927129 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0026 GB value = 0.0155 GB dict = 0.16 MB all = 0.0192 GB } __ZipSize{ index = 0.0003 GB value = 0.0054 GB dict = 0.16 MB all = 0.0062 GB } UnZip/Zip{ index = 10.3864 value = 2.8914 dict = 1.00 all = 3.0929 } Zip/UnZip{ index = 0.0963 value = 0.3458 dict = 1.00 all = 0.3233 } ---------------------------- total value len = 1.840782 GB avg = 0.123 KB (by entry num) total key len = 0.419372 GB avg = 0.028 KB total ukey len = 0.299552 GB avg = 0.020 KB total ukey num = 0.014977577 Billion total entry num = 0.014977577 Billion write speed all = 0.029266530 MB/sec (with seq num) write speed all = 0.027714984 MB/sec (without seq num) 2024/11/23-06:19:29.677690 140025146283584 [compaction_job.cc:1948] [default] [JOB 20901] Generated table #248270: 132447 keys, 6218085 bytes, temperature: kWarm 2024/11/23-06:19:29.677747 140025146283584 EVENT_LOG_v1 {"time_micros": 1732313969677716, "cf_name": "default", "job": 20901, "event": "table_file_creation", "file_number": 248270, "file_size": 6218085, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3771510047, "largest_seqno": 3831416638, "table_properties": {"data_size": 5368832, "index_size": 255040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3708516, "raw_average_key_size": 28, "raw_value_size": 15523698, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 132447, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313969, "oldest_key_time": 0, "file_creation_time": 1732313969, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248270, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:19:29.689430 140025146283584 [compaction_job.cc:703] job-20901: subcompact[0], size: 0.593197 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20901/att-00 [248254,248255,248256,248257,248258,248259,248260,248261,248262,248263,248264,248265,248266,248267,248268,248269,248270] 2024/11/23-06:19:29.793448 140025146283584 (Original Log Time 2024/11/23-06:18:31.676894) EVENT_LOG_v1 {"time_micros": 1732313911676856, "job": 20901, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [248215], "files_L3": [248136, 248137, 248138, 248139, 248140, 248141, 248142, 248143, 248144, 248145, 248146, 248147, 248148, 248149, 248150, 248239], "score": 0.997136, "input_data_size": 618479125, "oldest_snapshot_seqno": 3831769234} 2024/11/23-06:19:29.793452 140025146283584 (Original Log Time 2024/11/23-06:19:29.792609) [compaction_job.cc:2022] [default] [JOB 20901] Compacted 1@2 + 16@3 files to L3 => 593196772 bytes 2024/11/23-06:19:29.793453 140025146283584 (Original Log Time 2024/11/23-06:19:29.793379) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 17 0 0 0] max score 0.02, MB/sec: 10.7 rd, 10.2 wr, level 3, files in(1, 16) out(17 +0 blob) MB in(26.3, 563.5 +0.0 blob) out(565.7 +0.0 blob), read-write-amplify(43.9) write-amplify(21.5) OK, records in: 15071112, records dropped: 93535 output_compression: Snappy 2024/11/23-06:19:29.793458 140025146283584 (Original Log Time 2024/11/23-06:19:29.793426) EVENT_LOG_v1 {"time_micros": 1732313969793404, "cf": "default", "job": 20901, "event": "compaction_finished", "compaction_time_micros": 58012543, "compaction_time_cpu_micros": 23672719, "output_level": 3, "num_output_files": 17, "total_output_size": 593196772, "num_input_records": 15071112, "num_output_records": 14977577, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 410191935, "file_range_sync_nanos": 0, "file_fsync_nanos": 16521, "file_prepare_write_nanos": 37526, "lsm_state": [0, 0, 0, 17, 0, 0, 0]} 2024/11/23-06:19:29.918828 140025146283584 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732313911,"executesMs":58297,"compactionJobId":20901,"attempt":0,"compactionInputRawBytes":2179999128,"compactionInputZipBytes":618479125,"compactionOutputRawBytes":0,"compactionOutputZipBytes":593196772,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-f2c1cef9f2c0532ac33b8f426bdff84f-5930d10fe96c8f36-00"}}, response = 2024/11/23-06:19:29.918898 140025146283584 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20901/att-00: olev 3, work 58.298 s, result 4.897 ms, install 0.868 ms, input{raw 2.030 GiB zip 589.828 MiB}