2024/11/23-16:00:38.609588 139957273912896 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00: fork to child time = 0.008322 sec 2024/11/23-16:00:38.610402 139957273912896 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00 2024/11/23-16:00:38.617818 139957273912896 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00 2024/11/23-16:00:38.618718 139957273912896 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-16:00:38.618994 139957273912896 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-16:00:38.673309 139957273912896 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-30045/att-00/MANIFEST-206347 2024/11/23-16:00:38.709091 139957273912896 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-30045/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 268172, last_sequence is 4091045274, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 268027 2024/11/23-16:00:38.709102 139957273912896 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-16:00:38.709233 139957273912896 [version_set.cc:5721] Creating manifest 268172 2024/11/23-16:00:38.748832 139957273912896 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-16:00:38.772127 139957273912896 [compaction_job.cc:2437] [default] [JOB 30045] Compacting 2@2 + 22@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-16:00:38.772159 139957273912896 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 22464 Base level 2, inputs: [268166(27MB) 268170(9089KB)], [268064(38MB) 268106(39MB) 268107(36MB) 268108(35MB) 268109(34MB) 268110(34MB) 268111(33MB) 268112(34MB) 268113(34MB) 268114(35MB) 268115(38MB) 268116(35MB) 268117(36MB) 268118(33MB) 268119(35MB) 268120(36MB) 268121(35MB) 268122(36MB) 268123(35MB) 268124(35MB) 268125(36MB) 268126(37MB)] 2024/11/23-16:00:38.779374 139957273912896 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-16:00:38.779409 139957273912896 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-16:00:38.779429 139957273912896 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-16:00:38.779435 139957273912896 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-16:00:38.779447 139957273912896 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-16:00:39.455627 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268173.sst) /dev/shm/ToplingTemp/Topling-105319-yzbQUU 2024/11/23-16:00:40.530371 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268173.sst): first pass time = 1.07's, 124.891'MB/sec 2024/11/23-16:00:42.796815 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268173.sst): old prealloc_size = 147639500, real_size = 36367040 2024/11/23-16:00:42.797479 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268173.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928401 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0344 GB avg-val = 116.57 avg-zval = 37.02 2024/11/23-16:00:42.798318 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268173.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:00:43.654349 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268173.sst): second pass time = 2.26's, 47.781'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 46.922'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2716.617'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 298.090'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26330.244'MB/sec zip my value time = 2.26's, unzip length = 0.108'GB zip my value throughput = 47.781'MB/sec zip pipeline throughput = 70.821'MB/sec entries = 928401 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.57 avg-zval = 37.02 usrkeys = 928401 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.57 avg-zval = 37.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928401 bytes = 3017312 } seq expand size = 6498807 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.5148 value = 3.1489 dict = 1.00 all = 3.3356 } Zip/UnZip{ index = 0.0951 value = 0.3176 dict = 1.00 all = 0.2998 } ---------------------------- total value len = 8.963341 GB avg = 0.168 KB (by entry num) total key len = 1.466816 GB avg = 0.028 KB total ukey len = 1.016446 GB avg = 0.015 KB total ukey num = 0.066478836 Billion total entry num = 0.053286565 Billion write speed all = 0.408522107 MB/sec (with seq num) write speed all = 0.391825339 MB/sec (without seq num) 2024/11/23-16:00:43.704655 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268173: 928401 keys, 40237794 bytes, temperature: kWarm 2024/11/23-16:00:43.704744 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348843704690, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268173, "file_size": 40237794, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668893, "largest_seqno": 4091004391, "table_properties": {"data_size": 34368880, "index_size": 1765888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995228, "raw_average_key_size": 28, "raw_value_size": 108222634, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928401, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348839, "oldest_key_time": 0, "file_creation_time": 1732348839, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268173, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:00:43.747114 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268174.sst) /dev/shm/ToplingTemp/Topling-105319-0X4W4V 2024/11/23-16:00:45.298823 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268174.sst): first pass time = 1.55's, 86.499'MB/sec 2024/11/23-16:00:47.399571 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268174.sst): old prealloc_size = 147639500, real_size = 37083472 2024/11/23-16:00:47.400147 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268174.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928223 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0351 GB avg-val = 116.60 avg-zval = 37.81 2024/11/23-16:00:47.400962 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268174.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:00:48.218060 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268174.sst): second pass time = 2.10's, 51.535'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 50.467'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3184.246'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 315.139'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33429.677'MB/sec zip my value time = 2.10's, unzip length = 0.108'GB zip my value throughput = 51.535'MB/sec zip pipeline throughput = 51.535'MB/sec entries = 928223 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.60 avg-zval = 37.81 usrkeys = 928223 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.60 avg-zval = 37.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928223 bytes = 3364816 } seq expand size = 6497561 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0351 GB dict = 1.08 MB all = 0.0413 GB } UnZip/Zip{ index = 10.5834 value = 3.0837 dict = 1.00 all = 3.2494 } Zip/UnZip{ index = 0.0945 value = 0.3243 dict = 1.00 all = 0.3078 } ---------------------------- total value len = 9.078066 GB avg = 0.167 KB (by entry num) total key len = 1.492807 GB avg = 0.028 KB total ukey len = 1.035010 GB avg = 0.015 KB total ukey num = 0.067407059 Billion total entry num = 0.054214788 Billion write speed all = 0.413959567 MB/sec (with seq num) write speed all = 0.396974986 MB/sec (without seq num) 2024/11/23-16:00:48.272279 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268174: 928223 keys, 41305620 bytes, temperature: kWarm 2024/11/23-16:00:48.272375 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348848272323, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268174, "file_size": 41305620, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3736072251, "largest_seqno": 4091004334, "table_properties": {"data_size": 35097152, "index_size": 1754112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25990244, "raw_average_key_size": 28, "raw_value_size": 108227558, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928223, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348843, "oldest_key_time": 0, "file_creation_time": 1732348843, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268174, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:00:48.330838 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268175.sst) /dev/shm/ToplingTemp/Topling-105319-UlHMdX 2024/11/23-16:00:49.669203 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268175.sst): first pass time = 1.34's, 100.289'MB/sec 2024/11/23-16:00:51.679949 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268175.sst): old prealloc_size = 147639500, real_size = 33584288 2024/11/23-16:00:51.680673 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268175.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935724 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0316 GB avg-val = 115.44 avg-zval = 33.73 2024/11/23-16:00:51.681590 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268175.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:00:52.928356 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268175.sst): second pass time = 2.01's, 53.761'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.25's, 30.300'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2590.679'MB/sec (index lex order gen) rebuild zvType time = 1.05's, 0.223'MB/sec write SST data time = 0.20's, 190.593'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 33673.887'MB/sec zip my value time = 2.01's, unzip length = 0.108'GB zip my value throughput = 53.761'MB/sec zip pipeline throughput = 53.761'MB/sec entries = 935724 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.44 avg-zval = 33.73 usrkeys = 935724 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.44 avg-zval = 33.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935724 bytes = 3392016 } seq expand size = 6550068 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4463 value = 3.4228 dict = 1.00 all = 3.5483 } Zip/UnZip{ index = 0.0957 value = 0.2922 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 9.192634 GB avg = 0.167 KB (by entry num) total key len = 1.519007 GB avg = 0.028 KB total ukey len = 1.053725 GB avg = 0.015 KB total ukey num = 0.068342783 Billion total entry num = 0.055150512 Billion write speed all = 0.419394730 MB/sec (with seq num) write speed all = 0.402120190 MB/sec (without seq num) 2024/11/23-16:00:52.976446 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268175: 935724 keys, 37825636 bytes, temperature: kWarm 2024/11/23-16:00:52.976520 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348852976479, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268175, "file_size": 37825636, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4012776165, "table_properties": {"data_size": 31558672, "index_size": 1791488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26200272, "raw_average_key_size": 28, "raw_value_size": 108017496, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935724, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348848, "oldest_key_time": 0, "file_creation_time": 1732348848, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268175, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:00:53.018184 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268176.sst) /dev/shm/ToplingTemp/Topling-105319-GVIHXT 2024/11/23-16:00:54.212158 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268176.sst): first pass time = 1.19's, 112.416'MB/sec 2024/11/23-16:00:55.945364 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268176.sst): old prealloc_size = 147639500, real_size = 32343376 2024/11/23-16:00:55.946018 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268176.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930633 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.22 avg-zval = 32.60 2024/11/23-16:00:55.946838 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268176.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:00:56.942014 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268176.sst): second pass time = 1.73's, 62.432'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 36.576'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2805.275'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.271'MB/sec write SST data time = 0.14's, 264.989'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34825.129'MB/sec zip my value time = 1.73's, unzip length = 0.108'GB zip my value throughput = 62.432'MB/sec zip pipeline throughput = 103.469'MB/sec entries = 930633 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 32.60 usrkeys = 930633 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 32.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930633 bytes = 3257232 } seq expand size = 6514431 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5146 value = 3.5649 dict = 1.00 all = 3.6820 } Zip/UnZip{ index = 0.0951 value = 0.2805 dict = 1.00 all = 0.2716 } ---------------------------- total value len = 9.307308 GB avg = 0.166 KB (by entry num) total key len = 1.545065 GB avg = 0.028 KB total ukey len = 1.072338 GB avg = 0.015 KB total ukey num = 0.069273416 Billion total entry num = 0.056081145 Billion write speed all = 0.424838082 MB/sec (with seq num) write speed all = 0.407274803 MB/sec (without seq num) 2024/11/23-16:00:56.986346 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268176: 930633 keys, 36451962 bytes, temperature: kWarm 2024/11/23-16:00:56.986416 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348856986381, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268176, "file_size": 36451962, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3774999596, "largest_seqno": 4012776176, "table_properties": {"data_size": 30340352, "index_size": 1770176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26057724, "raw_average_key_size": 28, "raw_value_size": 108160066, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930633, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348853, "oldest_key_time": 0, "file_creation_time": 1732348853, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268176, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:00:57.025584 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268177.sst) /dev/shm/ToplingTemp/Topling-105319-oZ8duX 2024/11/23-16:00:58.112758 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268177.sst): first pass time = 1.09's, 123.461'MB/sec 2024/11/23-16:00:59.652627 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268177.sst): old prealloc_size = 147639500, real_size = 32446688 2024/11/23-16:00:59.653228 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268177.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933475 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.78 avg-zval = 32.59 2024/11/23-16:00:59.653866 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268177.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:00.606586 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268177.sst): second pass time = 1.54's, 70.224'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 37.513'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3090.328'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.24's, 151.457'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33970.637'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.224'MB/sec zip pipeline throughput = 70.224'MB/sec entries = 933475 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.78 avg-zval = 32.59 usrkeys = 933475 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.98 avg-zval = 32.59 TagRS{ kinds = RS_Key0_TagN bytes = 145960 } TagArray{ size = 692408 bytes = 2336896 } seq expand size = 4846856 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.4444 value = 3.5523 dict = 1.00 all = 3.7508 } Zip/UnZip{ index = 0.0957 value = 0.2815 dict = 1.00 all = 0.2666 } ---------------------------- total value len = 9.420236 GB avg = 0.165 KB (by entry num) total key len = 1.571202 GB avg = 0.028 KB total ukey len = 1.091007 GB avg = 0.016 KB total ukey num = 0.070206891 Billion total entry num = 0.057014620 Billion write speed all = 0.430220331 MB/sec (with seq num) write speed all = 0.412367272 MB/sec (without seq num) 2024/11/23-16:01:00.655069 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268177: 933475 keys, 35784198 bytes, temperature: kWarm 2024/11/23-16:01:00.655161 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348860655111, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268177, "file_size": 35784198, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776192, "table_properties": {"data_size": 30425616, "index_size": 1787520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26137300, "raw_average_key_size": 28, "raw_value_size": 108080514, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933475, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348857, "oldest_key_time": 0, "file_creation_time": 1732348857, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268177, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:00.717736 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268178.sst) /dev/shm/ToplingTemp/Topling-105319-Vchy7U 2024/11/23-16:01:01.926416 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268178.sst): first pass time = 1.21's, 111.050'MB/sec 2024/11/23-16:01:03.575514 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268178.sst): old prealloc_size = 147639500, real_size = 34954224 2024/11/23-16:01:03.576338 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268178.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081586 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.09 avg-zval = 30.17 2024/11/23-16:01:03.576393 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268178.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:04.260373 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268178.sst): second pass time = 1.65's, 63.060'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.68's, 52.172'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2572.213'MB/sec (index lex order gen) rebuild zvType time = 0.54's, 0.499'MB/sec write SST data time = 0.14's, 250.862'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.036'MB, throughput = 37299.384'MB/sec zip my value time = 1.65's, unzip length = 0.104'GB zip my value throughput = 63.060'MB/sec zip pipeline throughput = 63.060'MB/sec entries = 1081586 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.17 usrkeys = 1081586 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.17 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.04 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5164 value = 3.1855 dict = 1.00 all = 3.7564 } Zip/UnZip{ index = 0.0951 value = 0.3139 dict = 1.00 all = 0.2662 } ---------------------------- total value len = 9.524169 GB avg = 0.164 KB (by entry num) total key len = 1.601486 GB avg = 0.028 KB total ukey len = 1.112639 GB avg = 0.016 KB total ukey num = 0.071288477 Billion total entry num = 0.058096206 Billion write speed all = 0.435411537 MB/sec (with seq num) write speed all = 0.417222400 MB/sec (without seq num) 2024/11/23-16:01:04.311411 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268178: 1081586 keys, 35730414 bytes, temperature: kWarm 2024/11/23-16:01:04.311478 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348864311440, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268178, "file_size": 35730414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32626656, "index_size": 2056960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30284408, "raw_average_key_size": 28, "raw_value_size": 103933461, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081586, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348860, "oldest_key_time": 0, "file_creation_time": 1732348860, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268178, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:04.346623 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268179.sst) /dev/shm/ToplingTemp/Topling-105319-mfoN5T 2024/11/23-16:01:05.566086 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268179.sst): first pass time = 1.22's, 110.067'MB/sec 2024/11/23-16:01:06.953369 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268179.sst): old prealloc_size = 147639500, real_size = 31840848 2024/11/23-16:01:06.954070 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268179.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925652 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0298 GB avg-val = 117.00 avg-zval = 32.23 2024/11/23-16:01:06.954962 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268179.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:07.972433 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268179.sst): second pass time = 1.39's, 78.119'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 35.395'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2612.476'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.270'MB/sec write SST data time = 0.16's, 223.680'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37117.095'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 78.119'MB/sec zip pipeline throughput = 78.119'MB/sec entries = 925652 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.00 avg-zval = 32.23 usrkeys = 925652 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 32.23 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 859902 bytes = 3224640 } seq expand size = 6019314 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4451 value = 3.6297 dict = 1.00 all = 3.7211 } Zip/UnZip{ index = 0.0957 value = 0.2755 dict = 1.00 all = 0.2687 } ---------------------------- total value len = 9.638488 GB avg = 0.163 KB (by entry num) total key len = 1.627405 GB avg = 0.028 KB total ukey len = 1.131152 GB avg = 0.016 KB total ukey num = 0.072214129 Billion total entry num = 0.059021858 Billion write speed all = 0.440835785 MB/sec (with seq num) write speed all = 0.422359524 MB/sec (without seq num) 2024/11/23-16:01:08.023951 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268179: 925652 keys, 36069342 bytes, temperature: kWarm 2024/11/23-16:01:08.024036 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348868024000, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268179, "file_size": 36069342, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775885, "table_properties": {"data_size": 29836864, "index_size": 1772416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25918256, "raw_average_key_size": 28, "raw_value_size": 108299480, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925652, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348864, "oldest_key_time": 0, "file_creation_time": 1732348864, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268179, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:08.063728 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268180.sst) /dev/shm/ToplingTemp/Topling-105319-ZOHudX 2024/11/23-16:01:09.292513 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268180.sst): first pass time = 1.23's, 109.231'MB/sec 2024/11/23-16:01:10.757640 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268180.sst): old prealloc_size = 147639500, real_size = 32380672 2024/11/23-16:01:10.758350 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268180.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930646 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.22 avg-zval = 32.64 2024/11/23-16:01:10.759258 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268180.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:11.713955 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268180.sst): second pass time = 1.46's, 73.866'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.278'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2572.259'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.16's, 229.994'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38125.482'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 73.866'MB/sec zip pipeline throughput = 73.866'MB/sec entries = 930646 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 32.64 usrkeys = 930646 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 32.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930646 bytes = 3373600 } seq expand size = 6514522 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5132 value = 3.5605 dict = 1.00 all = 3.6666 } Zip/UnZip{ index = 0.0951 value = 0.2809 dict = 1.00 all = 0.2727 } ---------------------------- total value len = 9.753162 GB avg = 0.163 KB (by entry num) total key len = 1.653463 GB avg = 0.028 KB total ukey len = 1.149765 GB avg = 0.016 KB total ukey num = 0.073144775 Billion total entry num = 0.059952504 Billion write speed all = 0.446277319 MB/sec (with seq num) write speed all = 0.427512474 MB/sec (without seq num) 2024/11/23-16:01:11.763189 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268180: 930646 keys, 36605438 bytes, temperature: kWarm 2024/11/23-16:01:11.763251 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348871763217, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268180, "file_size": 36605438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775902, "table_properties": {"data_size": 30377392, "index_size": 1770432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058088, "raw_average_key_size": 28, "raw_value_size": 108159664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930646, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348868, "oldest_key_time": 0, "file_creation_time": 1732348868, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268180, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:11.802650 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268181.sst) /dev/shm/ToplingTemp/Topling-105319-f57oDW 2024/11/23-16:01:12.864752 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268181.sst): first pass time = 1.06's, 126.376'MB/sec 2024/11/23-16:01:14.289704 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268181.sst): old prealloc_size = 147639500, real_size = 32463904 2024/11/23-16:01:14.290414 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268181.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.97 avg-zval = 32.66 2024/11/23-16:01:14.291338 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268181.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:15.284199 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268181.sst): second pass time = 1.42's, 75.926'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 36.781'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2603.020'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.16's, 225.306'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36292.044'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 75.926'MB/sec zip pipeline throughput = 75.926'MB/sec entries = 932240 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.97 avg-zval = 32.66 usrkeys = 932240 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.97 avg-zval = 32.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932240 bytes = 3262848 } seq expand size = 6525680 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4433 value = 3.5511 dict = 1.00 all = 3.6694 } Zip/UnZip{ index = 0.0958 value = 0.2816 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 9.867803 GB avg = 0.162 KB (by entry num) total key len = 1.679565 GB avg = 0.028 KB total ukey len = 1.168410 GB avg = 0.016 KB total ukey num = 0.074077015 Billion total entry num = 0.060884744 Billion write speed all = 0.451720724 MB/sec (with seq num) write speed all = 0.432666754 MB/sec (without seq num) 2024/11/23-16:01:15.332847 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268181: 932240 keys, 36577510 bytes, temperature: kWarm 2024/11/23-16:01:15.332913 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348875332880, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268181, "file_size": 36577510, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 30445328, "index_size": 1785344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102720, "raw_average_key_size": 28, "raw_value_size": 108115060, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348871, "oldest_key_time": 0, "file_creation_time": 1732348871, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268181, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:15.370618 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268182.sst) /dev/shm/ToplingTemp/Topling-105319-O9jHyX 2024/11/23-16:01:16.561212 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268182.sst): first pass time = 1.19's, 112.736'MB/sec 2024/11/23-16:01:18.066014 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268182.sst): old prealloc_size = 147639500, real_size = 33145552 2024/11/23-16:01:18.066721 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268182.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925526 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.02 avg-zval = 33.66 2024/11/23-16:01:18.067645 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268182.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:19.003276 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268182.sst): second pass time = 1.50's, 72.010'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.980'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2576.958'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.16's, 235.820'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33818.131'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.010'MB/sec zip pipeline throughput = 72.010'MB/sec entries = 925526 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.02 avg-zval = 33.66 usrkeys = 925526 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.02 avg-zval = 33.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925526 bytes = 3470736 } seq expand size = 6478682 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5131 value = 3.4765 dict = 1.00 all = 3.5819 } Zip/UnZip{ index = 0.0951 value = 0.2876 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 9.982585 GB avg = 0.162 KB (by entry num) total key len = 1.705480 GB avg = 0.028 KB total ukey len = 1.186920 GB avg = 0.016 KB total ukey num = 0.075002541 Billion total entry num = 0.061810270 Billion write speed all = 0.457158110 MB/sec (with seq num) write speed all = 0.437817309 MB/sec (without seq num) 2024/11/23-16:01:19.052700 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268182: 925526 keys, 37471090 bytes, temperature: kWarm 2024/11/23-16:01:19.052761 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348879052728, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268182, "file_size": 37471090, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012774578, "table_properties": {"data_size": 31153280, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25914728, "raw_average_key_size": 28, "raw_value_size": 108303064, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925526, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348875, "oldest_key_time": 0, "file_creation_time": 1732348875, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268182, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:19.091463 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268183.sst) /dev/shm/ToplingTemp/Topling-105319-iRywQW 2024/11/23-16:01:20.298732 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268183.sst): first pass time = 1.21's, 111.179'MB/sec 2024/11/23-16:01:21.943921 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268183.sst): old prealloc_size = 147639500, real_size = 35343600 2024/11/23-16:01:21.944526 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268183.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929432 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0333 GB avg-val = 116.41 avg-zval = 35.86 2024/11/23-16:01:21.945330 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268183.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:22.820802 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268183.sst): second pass time = 1.64's, 65.793'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 45.121'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3041.079'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 312.842'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27569.325'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.793'MB/sec zip pipeline throughput = 78.193'MB/sec entries = 929432 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.41 avg-zval = 35.86 usrkeys = 929432 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.41 avg-zval = 35.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929432 bytes = 3369200 } seq expand size = 6506024 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4444 value = 3.2460 dict = 1.00 all = 3.3924 } Zip/UnZip{ index = 0.0957 value = 0.3081 dict = 1.00 all = 0.2948 } ---------------------------- total value len = 10.097284 GB avg = 0.161 KB (by entry num) total key len = 1.731504 GB avg = 0.028 KB total ukey len = 1.205509 GB avg = 0.016 KB total ukey num = 0.075931973 Billion total entry num = 0.062739702 Billion write speed all = 0.462593203 MB/sec (with seq num) write speed all = 0.442964508 MB/sec (without seq num) 2024/11/23-16:01:22.869521 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268183: 929432 keys, 39564736 bytes, temperature: kWarm 2024/11/23-16:01:22.869591 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348882869556, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268183, "file_size": 39564736, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33331296, "index_size": 1779776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26024096, "raw_average_key_size": 28, "raw_value_size": 108193716, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348879, "oldest_key_time": 0, "file_creation_time": 1732348879, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268183, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:22.919902 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268184.sst) /dev/shm/ToplingTemp/Topling-105319-aL9TqX 2024/11/23-16:01:23.879014 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268184.sst): first pass time = 0.96's, 139.949'MB/sec 2024/11/23-16:01:25.679660 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268184.sst): old prealloc_size = 147639500, real_size = 32683456 2024/11/23-16:01:25.680232 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268184.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928520 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.55 avg-zval = 33.05 2024/11/23-16:01:25.681102 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268184.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:26.753785 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268184.sst): second pass time = 1.80's, 60.140'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 34.465'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.775'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.275'MB/sec write SST data time = 0.23's, 160.976'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 15490.910'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.140'MB/sec zip pipeline throughput = 133.290'MB/sec entries = 928520 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 33.05 usrkeys = 928520 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 33.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928520 bytes = 3481968 } seq expand size = 6499640 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5150 value = 3.5268 dict = 1.00 all = 3.6257 } Zip/UnZip{ index = 0.0951 value = 0.2835 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 10.212003 GB avg = 0.160 KB (by entry num) total key len = 1.757503 GB avg = 0.028 KB total ukey len = 1.224079 GB avg = 0.016 KB total ukey num = 0.076860493 Billion total entry num = 0.063668222 Billion write speed all = 0.468024313 MB/sec (with seq num) write speed all = 0.448108185 MB/sec (without seq num) 2024/11/23-16:01:26.800968 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268184: 928520 keys, 37018696 bytes, temperature: kWarm 2024/11/23-16:01:26.801056 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348886801020, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268184, "file_size": 37018696, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30685040, "index_size": 1766080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25998560, "raw_average_key_size": 28, "raw_value_size": 108219308, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928520, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348882, "oldest_key_time": 0, "file_creation_time": 1732348882, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268184, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:26.851918 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268185.sst) /dev/shm/ToplingTemp/Topling-105319-CwtcOV 2024/11/23-16:01:27.791763 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268185.sst): first pass time = 0.94's, 142.814'MB/sec 2024/11/23-16:01:29.311482 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268185.sst): old prealloc_size = 147639500, real_size = 35128608 2024/11/23-16:01:29.312075 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268185.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925677 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 116.99 avg-zval = 35.78 2024/11/23-16:01:29.312803 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268185.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:30.131509 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268185.sst): second pass time = 1.52's, 71.295'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 47.460'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3107.214'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 300.787'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29981.964'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.295'MB/sec zip pipeline throughput = 71.295'MB/sec entries = 925677 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.99 avg-zval = 35.78 usrkeys = 925677 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.61 avg-zval = 35.78 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 742680 bytes = 2785072 } seq expand size = 5198760 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4439 value = 3.2695 dict = 1.00 all = 3.4488 } Zip/UnZip{ index = 0.0958 value = 0.3059 dict = 1.00 all = 0.2900 } ---------------------------- total value len = 10.325501 GB avg = 0.160 KB (by entry num) total key len = 1.783422 GB avg = 0.028 KB total ukey len = 1.242593 GB avg = 0.016 KB total ukey num = 0.077786170 Billion total entry num = 0.064593899 Billion write speed all = 0.473413167 MB/sec (with seq num) write speed all = 0.453210145 MB/sec (without seq num) 2024/11/23-16:01:30.181220 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268185: 925677 keys, 38917110 bytes, temperature: kWarm 2024/11/23-16:01:30.181291 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348890181255, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268185, "file_size": 38917110, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 33124304, "index_size": 1772672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25918956, "raw_average_key_size": 28, "raw_value_size": 108298790, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925677, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348886, "oldest_key_time": 0, "file_creation_time": 1732348886, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268185, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:30.229754 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268186.sst) /dev/shm/ToplingTemp/Topling-105319-lngh4U 2024/11/23-16:01:31.227315 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268186.sst): first pass time = 1.00's, 134.552'MB/sec 2024/11/23-16:01:32.548190 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268186.sst): old prealloc_size = 147639500, real_size = 31958128 2024/11/23-16:01:32.548754 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268186.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0299 GB avg-val = 115.79 avg-zval = 32.08 2024/11/23-16:01:32.549596 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268186.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:33.397750 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268186.sst): second pass time = 1.32's, 81.874'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.603'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3241.058'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 283.934'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34653.144'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.874'MB/sec zip pipeline throughput = 81.874'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 32.08 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5146 value = 3.6088 dict = 1.00 all = 3.7084 } Zip/UnZip{ index = 0.0951 value = 0.2771 dict = 1.00 all = 0.2697 } ---------------------------- total value len = 10.440116 GB avg = 0.159 KB (by entry num) total key len = 1.809558 GB avg = 0.028 KB total ukey len = 1.261262 GB avg = 0.016 KB total ukey num = 0.078719627 Billion total entry num = 0.065527356 Billion write speed all = 0.478854891 MB/sec (with seq num) write speed all = 0.458362530 MB/sec (without seq num) 2024/11/23-16:01:33.443255 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268186: 933457 keys, 36192438 bytes, temperature: kWarm 2024/11/23-16:01:33.443324 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348893443289, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268186, "file_size": 36192438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 29949024, "index_size": 1775552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348890, "oldest_key_time": 0, "file_creation_time": 1732348890, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268186, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:33.484167 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268187.sst) /dev/shm/ToplingTemp/Topling-105319-QwXP4W 2024/11/23-16:01:34.471413 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268187.sst): first pass time = 0.99's, 135.956'MB/sec 2024/11/23-16:01:35.825751 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268187.sst): old prealloc_size = 147639500, real_size = 33122544 2024/11/23-16:01:35.826343 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268187.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932384 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0311 GB avg-val = 115.95 avg-zval = 33.35 2024/11/23-16:01:35.827205 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268187.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:36.677990 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268187.sst): second pass time = 1.35's, 79.870'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.831'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3145.031'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 296.103'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25729.723'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.870'MB/sec zip pipeline throughput = 79.870'MB/sec entries = 932384 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.95 avg-zval = 33.35 usrkeys = 932384 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.95 avg-zval = 33.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932384 bytes = 3379904 } seq expand size = 6526688 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3783 value = 3.4771 dict = 1.00 all = 3.5932 } Zip/UnZip{ index = 0.0964 value = 0.2876 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 10.554754 GB avg = 0.159 KB (by entry num) total key len = 1.835665 GB avg = 0.028 KB total ukey len = 1.279910 GB avg = 0.016 KB total ukey num = 0.079652011 Billion total entry num = 0.066459740 Billion write speed all = 0.484294667 MB/sec (with seq num) write speed all = 0.463513386 MB/sec (without seq num) 2024/11/23-16:01:36.725817 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268187: 932384 keys, 37352854 bytes, temperature: kWarm 2024/11/23-16:01:36.725889 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348896725854, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268187, "file_size": 37352854, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 31092448, "index_size": 1796800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26106752, "raw_average_key_size": 28, "raw_value_size": 108111048, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932384, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348893, "oldest_key_time": 0, "file_creation_time": 1732348893, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268187, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:36.779535 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268188.sst) /dev/shm/ToplingTemp/Topling-105319-pxzvJX 2024/11/23-16:01:37.828689 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268188.sst): first pass time = 1.05's, 127.934'MB/sec 2024/11/23-16:01:39.206539 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268188.sst): old prealloc_size = 147639500, real_size = 34863872 2024/11/23-16:01:39.207132 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268188.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0329 GB avg-val = 117.01 avg-zval = 35.50 2024/11/23-16:01:39.210134 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268188.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:40.053719 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268188.sst): second pass time = 1.38's, 78.648'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.260'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3108.561'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 303.438'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34791.834'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.648'MB/sec zip pipeline throughput = 78.648'MB/sec entries = 925568 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.01 avg-zval = 35.50 usrkeys = 925568 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.01 avg-zval = 35.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925568 bytes = 3470896 } seq expand size = 6478976 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4419 value = 3.2959 dict = 1.00 all = 3.4248 } Zip/UnZip{ index = 0.0958 value = 0.3034 dict = 1.00 all = 0.2920 } ---------------------------- total value len = 10.669535 GB avg = 0.158 KB (by entry num) total key len = 1.861581 GB avg = 0.028 KB total ukey len = 1.298421 GB avg = 0.016 KB total ukey num = 0.080577579 Billion total entry num = 0.067385308 Billion write speed all = 0.489729352 MB/sec (with seq num) write speed all = 0.468661435 MB/sec (without seq num) 2024/11/23-16:01:40.102907 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268188: 925568 keys, 39189730 bytes, temperature: kWarm 2024/11/23-16:01:40.102999 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348900102943, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268188, "file_size": 39189730, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32859504, "index_size": 1772800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915904, "raw_average_key_size": 28, "raw_value_size": 108301824, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348896, "oldest_key_time": 0, "file_creation_time": 1732348896, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268188, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:40.144353 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268189.sst) /dev/shm/ToplingTemp/Topling-105319-SaPO9V 2024/11/23-16:01:41.178474 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268189.sst): first pass time = 1.03's, 129.794'MB/sec 2024/11/23-16:01:42.501890 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268189.sst): old prealloc_size = 147639500, real_size = 32664400 2024/11/23-16:01:42.502449 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268189.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931651 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.06 avg-zval = 32.91 2024/11/23-16:01:42.503289 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268189.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:43.357468 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268189.sst): second pass time = 1.32's, 81.755'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.122'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.092'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 291.980'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29791.931'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.755'MB/sec zip pipeline throughput = 81.755'MB/sec entries = 931651 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 32.91 usrkeys = 931651 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 32.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931651 bytes = 3377248 } seq expand size = 6521557 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5128 value = 3.5269 dict = 1.00 all = 3.6381 } Zip/UnZip{ index = 0.0951 value = 0.2835 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 10.784188 GB avg = 0.158 KB (by entry num) total key len = 1.887667 GB avg = 0.028 KB total ukey len = 1.317054 GB avg = 0.016 KB total ukey num = 0.081509230 Billion total entry num = 0.068316959 Billion write speed all = 0.495165663 MB/sec (with seq num) write speed all = 0.473809225 MB/sec (without seq num) 2024/11/23-16:01:43.404889 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268189: 931651 keys, 36892714 bytes, temperature: kWarm 2024/11/23-16:01:43.404962 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348903404925, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268189, "file_size": 36892714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 30658880, "index_size": 1772416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26086228, "raw_average_key_size": 28, "raw_value_size": 108131578, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931651, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348900, "oldest_key_time": 0, "file_creation_time": 1732348900, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268189, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:43.443810 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268190.sst) /dev/shm/ToplingTemp/Topling-105319-glqfcX 2024/11/23-16:01:44.481592 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268190.sst): first pass time = 1.04's, 129.336'MB/sec 2024/11/23-16:01:45.803410 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268190.sst): old prealloc_size = 147639500, real_size = 34547696 2024/11/23-16:01:45.804010 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268190.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923008 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0326 GB avg-val = 117.41 avg-zval = 35.28 2024/11/23-16:01:45.804867 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268190.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:46.646261 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268190.sst): second pass time = 1.32's, 82.039'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.115'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3054.671'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 311.171'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32827.285'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.039'MB/sec zip pipeline throughput = 82.039'MB/sec entries = 923008 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.41 avg-zval = 35.28 usrkeys = 923008 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.41 avg-zval = 35.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923008 bytes = 3461296 } seq expand size = 6461056 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.5136 value = 3.3283 dict = 1.00 all = 3.4533 } Zip/UnZip{ index = 0.0951 value = 0.3005 dict = 1.00 all = 0.2896 } ---------------------------- total value len = 10.899023 GB avg = 0.157 KB (by entry num) total key len = 1.913512 GB avg = 0.028 KB total ukey len = 1.335514 GB avg = 0.016 KB total ukey num = 0.082432238 Billion total entry num = 0.069239967 Billion write speed all = 0.500598501 MB/sec (with seq num) write speed all = 0.478956303 MB/sec (without seq num) 2024/11/23-16:01:46.694361 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268190: 923008 keys, 38866888 bytes, temperature: kWarm 2024/11/23-16:01:46.694433 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348906694396, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268190, "file_size": 38866888, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 32560928, "index_size": 1755840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844224, "raw_average_key_size": 28, "raw_value_size": 108373600, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348903, "oldest_key_time": 0, "file_creation_time": 1732348903, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268190, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:46.732483 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268191.sst) /dev/shm/ToplingTemp/Topling-105319-SmSwEW 2024/11/23-16:01:47.640520 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268191.sst): first pass time = 0.91's, 147.818'MB/sec 2024/11/23-16:01:48.980426 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268191.sst): old prealloc_size = 147639500, real_size = 32969504 2024/11/23-16:01:48.981047 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268191.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930363 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.26 avg-zval = 33.26 2024/11/23-16:01:48.981882 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268191.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:49.838133 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268191.sst): second pass time = 1.34's, 80.773'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.501'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3033.294'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 298.998'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32190.769'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.773'MB/sec zip pipeline throughput = 80.773'MB/sec entries = 930363 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.26 avg-zval = 33.26 usrkeys = 930363 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.26 avg-zval = 33.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930363 bytes = 3488880 } seq expand size = 6512541 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.3757 value = 3.4957 dict = 1.00 all = 3.5974 } Zip/UnZip{ index = 0.0964 value = 0.2861 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 11.013703 GB avg = 0.157 KB (by entry num) total key len = 1.939562 GB avg = 0.028 KB total ukey len = 1.354121 GB avg = 0.016 KB total ukey num = 0.083362601 Billion total entry num = 0.070170330 Billion write speed all = 0.506033867 MB/sec (with seq num) write speed all = 0.484103603 MB/sec (without seq num) 2024/11/23-16:01:49.883915 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268191: 930363 keys, 37309894 bytes, temperature: kWarm 2024/11/23-16:01:49.884009 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348909883950, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268191, "file_size": 37309894, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 30943376, "index_size": 1793344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050164, "raw_average_key_size": 28, "raw_value_size": 108167598, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930363, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348906, "oldest_key_time": 0, "file_creation_time": 1732348906, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268191, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:49.920480 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268192.sst) /dev/shm/ToplingTemp/Topling-105319-MFVVAV 2024/11/23-16:01:50.914356 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268192.sst): first pass time = 0.99's, 135.049'MB/sec 2024/11/23-16:01:52.312692 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268192.sst): old prealloc_size = 147639500, real_size = 34918864 2024/11/23-16:01:52.313252 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268192.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928770 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0329 GB avg-val = 116.51 avg-zval = 35.43 2024/11/23-16:01:52.313738 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268192.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:53.092525 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268192.sst): second pass time = 1.40's, 77.430'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 48.152'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3305.751'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.355'MB/sec write SST data time = 0.13's, 299.639'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34841.029'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.430'MB/sec zip pipeline throughput = 77.430'MB/sec entries = 928770 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.51 avg-zval = 35.43 usrkeys = 928770 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.02 avg-zval = 35.43 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 464912 bytes = 1627200 } seq expand size = 3254384 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4429 value = 3.2884 dict = 1.00 all = 3.5744 } Zip/UnZip{ index = 0.0958 value = 0.3041 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 11.125169 GB avg = 0.156 KB (by entry num) total key len = 1.965567 GB avg = 0.028 KB total ukey len = 1.372697 GB avg = 0.016 KB total ukey num = 0.084291371 Billion total entry num = 0.071099100 Billion write speed all = 0.511339362 MB/sec (with seq num) write speed all = 0.489121655 MB/sec (without seq num) 2024/11/23-16:01:53.140333 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268192: 928770 keys, 37549411 bytes, temperature: kWarm 2024/11/23-16:01:53.140404 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348913140368, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268192, "file_size": 37549411, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 32907712, "index_size": 1778752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005560, "raw_average_key_size": 28, "raw_value_size": 108212220, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928770, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348909, "oldest_key_time": 0, "file_creation_time": 1732348909, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268192, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:53.184338 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268193.sst) /dev/shm/ToplingTemp/Topling-105319-N9UZtT 2024/11/23-16:01:54.184496 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268193.sst): first pass time = 1.00's, 134.203'MB/sec 2024/11/23-16:01:55.686495 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268193.sst): old prealloc_size = 147639500, real_size = 36864704 2024/11/23-16:01:55.687063 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268193.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926848 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.81 avg-zval = 37.61 2024/11/23-16:01:55.687300 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268193.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:56.400408 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268193.sst): second pass time = 1.50's, 72.120'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.719'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3309.376'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.392'MB/sec write SST data time = 0.12's, 311.821'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33656.655'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.120'MB/sec zip pipeline throughput = 72.120'MB/sec entries = 926848 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.81 avg-zval = 37.61 usrkeys = 926848 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.98 avg-zval = 37.61 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4465 value = 3.1059 dict = 1.00 all = 3.4998 } Zip/UnZip{ index = 0.0957 value = 0.3220 dict = 1.00 all = 0.2857 } ---------------------------- total value len = 11.234516 GB avg = 0.156 KB (by entry num) total key len = 1.991519 GB avg = 0.028 KB total ukey len = 1.391234 GB avg = 0.016 KB total ukey num = 0.085218219 Billion total entry num = 0.072025948 Billion write speed all = 0.516557540 MB/sec (with seq num) write speed all = 0.494053111 MB/sec (without seq num) 2024/11/23-16:01:56.448856 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268193: 926848 keys, 38350147 bytes, temperature: kWarm 2024/11/23-16:01:56.448925 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348916448891, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268193, "file_size": 38350147, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34858352, "index_size": 1774464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951744, "raw_average_key_size": 28, "raw_value_size": 108266116, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926848, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348913, "oldest_key_time": 0, "file_creation_time": 1732348913, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268193, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:56.496760 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268194.sst) /dev/shm/ToplingTemp/Topling-105319-WLR4VV 2024/11/23-16:01:57.393365 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268194.sst): first pass time = 0.90's, 149.702'MB/sec 2024/11/23-16:01:58.729419 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268194.sst): old prealloc_size = 147639500, real_size = 35559056 2024/11/23-16:01:58.729950 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268194.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925008 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0336 GB avg-val = 117.10 avg-zval = 36.29 2024/11/23-16:01:58.730748 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268194.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:01:59.550420 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268194.sst): second pass time = 1.34's, 81.118'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.070'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3547.286'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 309.814'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27590.781'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.118'MB/sec zip pipeline throughput = 81.118'MB/sec entries = 925008 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 36.29 usrkeys = 925008 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 36.29 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 859723 bytes = 2901584 } seq expand size = 6018061 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5111 value = 3.2269 dict = 1.00 all = 3.4010 } Zip/UnZip{ index = 0.0951 value = 0.3099 dict = 1.00 all = 0.2940 } ---------------------------- total value len = 11.348852 GB avg = 0.156 KB (by entry num) total key len = 2.017419 GB avg = 0.028 KB total ukey len = 1.409734 GB avg = 0.016 KB total ukey num = 0.086143227 Billion total entry num = 0.072950956 Billion write speed all = 0.521970392 MB/sec (with seq num) write speed all = 0.499179749 MB/sec (without seq num) 2024/11/23-16:01:59.600022 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268194: 925008 keys, 39464630 bytes, temperature: kWarm 2024/11/23-16:01:59.600098 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348919600060, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268194, "file_size": 39464630, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4091003994, "table_properties": {"data_size": 33567552, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900224, "raw_average_key_size": 28, "raw_value_size": 108317608, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348916, "oldest_key_time": 0, "file_creation_time": 1732348916, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268194, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:01:59.639853 139957273912896 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268195.sst) /dev/shm/ToplingTemp/Topling-105319-Ji2UHX 2024/11/23-16:01:59.881627 139957273912896 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268195.sst): first pass time = 0.24's, 116.550'MB/sec 2024/11/23-16:02:00.166338 139957273912896 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268195.sst): old prealloc_size = 147639500, real_size = 8379328 2024/11/23-16:02:00.166519 139957273912896 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268195.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 195144 prefix = 4 raw-key = 0.0031 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0227 GB zip-val = 0.0080 GB avg-val = 116.38 avg-zval = 40.77 2024/11/23-16:02:00.166713 139957273912896 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268195.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.028 GB 2024/11/23-16:02:00.334301 139957273912896 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00/268195.sst): second pass time = 0.28's, 79.968'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 54.660'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2393.600'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.347'MB/sec write SST data time = 0.03's, 336.333'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.237'MB, throughput = 8286.373'MB/sec zip my value time = 0.28's, unzip length = 0.023'GB zip my value throughput = 79.968'MB/sec zip pipeline throughput = 79.968'MB/sec entries = 195144 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.38 avg-zval = 40.77 usrkeys = 195144 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.38 avg-zval = 40.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 195144 bytes = 609840 } seq expand size = 1366008 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0039 GB value = 0.0227 GB dict = 0.24 MB all = 0.0282 GB } __ZipSize{ index = 0.0004 GB value = 0.0080 GB dict = 0.24 MB all = 0.0092 GB } UnZip/Zip{ index = 10.4030 value = 2.8548 dict = 1.00 all = 3.0692 } Zip/UnZip{ index = 0.0961 value = 0.3503 dict = 1.00 all = 0.3258 } ---------------------------- total value len = 11.372928 GB avg = 0.155 KB (by entry num) total key len = 2.022883 GB avg = 0.028 KB total ukey len = 1.413637 GB avg = 0.016 KB total ukey num = 0.086338371 Billion total entry num = 0.073146100 Billion write speed all = 0.523107976 MB/sec (with seq num) write speed all = 0.500257068 MB/sec (without seq num) 2024/11/23-16:02:00.351375 139957273912896 [compaction_job.cc:1948] [default] [JOB 30045] Generated table #268195: 195144 keys, 9179815 bytes, temperature: kWarm 2024/11/23-16:02:00.351436 139957273912896 EVENT_LOG_v1 {"time_micros": 1732348920351403, "cf_name": "default", "job": 30045, "event": "table_file_creation", "file_number": 268195, "file_size": 9179815, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4072856654, "largest_seqno": 4091004053, "table_properties": {"data_size": 7955184, "index_size": 375168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5464032, "raw_average_key_size": 28, "raw_value_size": 22710508, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 195144, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348919, "oldest_key_time": 0, "file_creation_time": 1732348919, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268195, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:02:00.368065 139957273912896 [compaction_job.cc:703] job-30045: subcompact[0], size: 0.839908 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30045/att-00 [268173,268174,268175,268176,268177,268178,268179,268180,268181,268182,268183,268184,268185,268186,268187,268188,268189,268190,268191,268192,268193,268194,268195] 2024/11/23-16:02:00.544531 139957273912896 (Original Log Time 2024/11/23-16:00:38.772231) EVENT_LOG_v1 {"time_micros": 1732348838772178, "job": 30045, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [268166, 268170], "files_L3": [268064, 268106, 268107, 268108, 268109, 268110, 268111, 268112, 268113, 268114, 268115, 268116, 268117, 268118, 268119, 268120, 268121, 268122, 268123, 268124, 268125, 268126], "score": 0.991354, "input_data_size": 865112859, "oldest_snapshot_seqno": 4091045092} 2024/11/23-16:02:00.544535 139957273912896 (Original Log Time 2024/11/23-16:02:00.542824) [compaction_job.cc:2022] [default] [JOB 30045] Compacted 2@2 + 22@3 files to L3 => 839908067 bytes 2024/11/23-16:02:00.544536 139957273912896 (Original Log Time 2024/11/23-16:02:00.544447) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 10.6 rd, 10.3 wr, level 3, files in(2, 22) out(23 +0 blob) MB in(36.0, 789.0 +0.0 blob) out(801.0 +0.0 blob), read-write-amplify(45.1) write-amplify(22.2) OK, records in: 20903266, records dropped: 115330 output_compression: Snappy 2024/11/23-16:02:00.544540 139957273912896 (Original Log Time 2024/11/23-16:02:00.544505) EVENT_LOG_v1 {"time_micros": 1732348920544481, "cf": "default", "job": 30045, "event": "compaction_finished", "compaction_time_micros": 81595840, "compaction_time_cpu_micros": 34004653, "output_level": 3, "num_output_files": 23, "total_output_size": 839908067, "num_input_records": 20903266, "num_output_records": 20787936, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 622122607, "file_range_sync_nanos": 0, "file_fsync_nanos": 15308, "file_prepare_write_nanos": 66283, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-16:02:00.682349 139957273912896 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732348838,"executesMs":81925,"compactionJobId":30045,"attempt":0,"compactionInputRawBytes":2997343029,"compactionInputZipBytes":865112859,"compactionOutputRawBytes":0,"compactionOutputZipBytes":839908067,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-43c736c0f9b6e7ff436e5c14cdbf0a20-83121e71a71792f6-00"}}, response = 2024/11/23-16:02:00.682444 139957273912896 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-30045/att-00: olev 3, work 81.925 s, result 7.678 ms, install 1.731 ms, input{raw 2.791 GiB zip 825.036 MiB}