2024/11/23-16:05:11.028718 139957659977280 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00: fork to child time = 0.007321 sec 2024/11/23-16:05:11.029467 139957659977280 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00 2024/11/23-16:05:11.036649 139957659977280 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00 2024/11/23-16:05:11.037527 139957659977280 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-16:05:11.037743 139957659977280 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-16:05:11.091293 139957659977280 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-30146/att-00/MANIFEST-206347 2024/11/23-16:05:11.113163 139957659977280 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-30146/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 268388, last_sequence is 4094137796, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 268241 2024/11/23-16:05:11.113171 139957659977280 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-16:05:11.113291 139957659977280 [version_set.cc:5721] Creating manifest 268388 2024/11/23-16:05:11.163221 139957659977280 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-16:05:11.196135 139957659977280 [compaction_job.cc:2437] [default] [JOB 30146] Compacting 1@2 + 23@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-16:05:11.196164 139957659977280 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 22540 Base level 2, inputs: [268374(25MB)], [268342(38MB) 268343(39MB) 268344(3509KB) 268309(35MB) 268310(34MB) 268311(33MB) 268312(34MB) 268313(34MB) 268314(35MB) 268315(34MB) 268316(35MB) 268317(37MB) 268318(35MB) 268319(37MB) 268320(34MB) 268321(35MB) 268322(37MB) 268323(35MB) 268324(37MB) 268325(35MB) 268326(35MB) 268327(36MB) 268328(37MB)] 2024/11/23-16:05:11.204742 139957659977280 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-16:05:11.204783 139957659977280 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-16:05:11.204810 139957659977280 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-16:05:11.204820 139957659977280 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-16:05:11.204834 139957659977280 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-16:05:11.912788 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268389.sst) /dev/shm/ToplingTemp/Topling-106742-ulhK0u 2024/11/23-16:05:13.017037 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268389.sst): first pass time = 1.10's, 121.553'MB/sec 2024/11/23-16:05:15.310731 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268389.sst): old prealloc_size = 147639500, real_size = 36581920 2024/11/23-16:05:15.311455 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268389.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927208 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0346 GB avg-val = 116.75 avg-zval = 37.30 2024/11/23-16:05:15.312276 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268389.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:16.364137 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268389.sst): second pass time = 2.29's, 47.215'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 38.400'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2521.945'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.26's, 153.372'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35456.270'MB/sec zip my value time = 2.29's, unzip length = 0.108'GB zip my value throughput = 47.215'MB/sec zip pipeline throughput = 97.240'MB/sec entries = 927208 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 37.30 usrkeys = 927208 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 37.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927208 bytes = 3013440 } seq expand size = 6490456 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.5162 value = 3.1300 dict = 1.00 all = 3.3181 } Zip/UnZip{ index = 0.0951 value = 0.3195 dict = 1.00 all = 0.3014 } ---------------------------- total value len = 8.963366 GB avg = 0.168 KB (by entry num) total key len = 1.466783 GB avg = 0.028 KB total ukey len = 1.016422 GB avg = 0.015 KB total ukey num = 0.066477643 Billion total entry num = 0.053285372 Billion write speed all = 0.404204339 MB/sec (with seq num) write speed all = 0.387684400 MB/sec (without seq num) 2024/11/23-16:05:16.417241 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268389: 927208 keys, 40449748 bytes, temperature: kWarm 2024/11/23-16:05:16.417353 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349116417278, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268389, "file_size": 40449748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668893, "largest_seqno": 4094023981, "table_properties": {"data_size": 34586512, "index_size": 1763392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961824, "raw_average_key_size": 28, "raw_value_size": 108256016, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927208, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349111, "oldest_key_time": 0, "file_creation_time": 1732349111, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268389, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:16.492455 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268390.sst) /dev/shm/ToplingTemp/Topling-106742-559qJy 2024/11/23-16:05:18.087210 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268390.sst): first pass time = 1.59's, 84.165'MB/sec 2024/11/23-16:05:19.659739 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268390.sst): old prealloc_size = 147639500, real_size = 36700880 2024/11/23-16:05:19.660451 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268390.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928305 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0347 GB avg-val = 116.58 avg-zval = 37.40 2024/11/23-16:05:19.661266 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268390.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:20.653086 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268390.sst): second pass time = 1.57's, 68.887'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 40.964'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2542.662'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.15's, 264.309'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33149.770'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.887'MB/sec zip pipeline throughput = 71.374'MB/sec entries = 928305 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.58 avg-zval = 37.40 usrkeys = 928305 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.58 avg-zval = 37.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928305 bytes = 3133040 } seq expand size = 6498135 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0407 GB } UnZip/Zip{ index = 10.5828 value = 3.1176 dict = 1.00 all = 3.2985 } Zip/UnZip{ index = 0.0945 value = 0.3208 dict = 1.00 all = 0.3032 } ---------------------------- total value len = 9.078090 GB avg = 0.167 KB (by entry num) total key len = 1.492775 GB avg = 0.028 KB total ukey len = 1.034988 GB avg = 0.015 KB total ukey num = 0.067405948 Billion total entry num = 0.054213677 Billion write speed all = 0.409589491 MB/sec (with seq num) write speed all = 0.392784545 MB/sec (without seq num) 2024/11/23-16:05:20.703721 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268390: 928305 keys, 40691032 bytes, temperature: kWarm 2024/11/23-16:05:20.703780 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349120703747, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268390, "file_size": 40691032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3967285072, "largest_seqno": 4094024050, "table_properties": {"data_size": 34714240, "index_size": 1754368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25992540, "raw_average_key_size": 28, "raw_value_size": 108225310, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928305, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349116, "oldest_key_time": 0, "file_creation_time": 1732349116, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268390, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:20.740791 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268391.sst) /dev/shm/ToplingTemp/Topling-106742-WiWdFv 2024/11/23-16:05:21.871351 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268391.sst): first pass time = 1.13's, 118.721'MB/sec 2024/11/23-16:05:23.234001 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268391.sst): old prealloc_size = 147639500, real_size = 34031600 2024/11/23-16:05:23.234699 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268391.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934568 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0320 GB avg-val = 115.61 avg-zval = 34.25 2024/11/23-16:05:23.236957 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268391.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:24.397647 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268391.sst): second pass time = 1.36's, 79.406'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.16's, 32.888'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2644.647'MB/sec (index lex order gen) rebuild zvType time = 1.00's, 0.234'MB/sec write SST data time = 0.17's, 229.807'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34931.299'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.406'MB/sec zip pipeline throughput = 79.406'MB/sec entries = 934568 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.61 avg-zval = 34.25 usrkeys = 934568 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.61 avg-zval = 34.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934568 bytes = 3387824 } seq expand size = 6541976 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4457 value = 3.3757 dict = 1.00 all = 3.5071 } Zip/UnZip{ index = 0.0957 value = 0.2962 dict = 1.00 all = 0.2851 } ---------------------------- total value len = 9.192682 GB avg = 0.167 KB (by entry num) total key len = 1.518943 GB avg = 0.028 KB total ukey len = 1.053680 GB avg = 0.015 KB total ukey num = 0.068340516 Billion total entry num = 0.055148245 Billion write speed all = 0.414983304 MB/sec (with seq num) write speed all = 0.397891144 MB/sec (without seq num) 2024/11/23-16:05:24.446056 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268391: 934568 keys, 38270130 bytes, temperature: kWarm 2024/11/23-16:05:24.446127 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349124446089, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268391, "file_size": 38270130, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4012776165, "table_properties": {"data_size": 32008416, "index_size": 1789376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26167904, "raw_average_key_size": 28, "raw_value_size": 108049944, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349120, "oldest_key_time": 0, "file_creation_time": 1732349120, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268391, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:24.483960 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268392.sst) /dev/shm/ToplingTemp/Topling-106742-fxaGJy 2024/11/23-16:05:25.610587 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268392.sst): first pass time = 1.13's, 119.137'MB/sec 2024/11/23-16:05:27.137238 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268392.sst): old prealloc_size = 147639500, real_size = 31673104 2024/11/23-16:05:27.137923 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268392.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933552 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0297 GB avg-val = 115.77 avg-zval = 31.78 2024/11/23-16:05:27.138798 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268392.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:28.079869 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268392.sst): second pass time = 1.53's, 70.835'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 37.970'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2688.556'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.22's, 166.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33820.772'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.835'MB/sec zip pipeline throughput = 70.835'MB/sec entries = 933552 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.77 avg-zval = 31.78 usrkeys = 933552 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.77 avg-zval = 31.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933552 bytes = 3267440 } seq expand size = 6534864 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5137 value = 3.6435 dict = 1.00 all = 3.7501 } Zip/UnZip{ index = 0.0951 value = 0.2745 dict = 1.00 all = 0.2667 } ---------------------------- total value len = 9.307295 GB avg = 0.166 KB (by entry num) total key len = 1.545083 GB avg = 0.028 KB total ukey len = 1.072351 GB avg = 0.015 KB total ukey num = 0.069274068 Billion total entry num = 0.056081797 Billion write speed all = 0.420376288 MB/sec (with seq num) write speed all = 0.402997271 MB/sec (without seq num) 2024/11/23-16:05:28.125152 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268392: 933552 keys, 35790908 bytes, temperature: kWarm 2024/11/23-16:05:28.125220 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349128125183, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268392, "file_size": 35790908, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776176, "table_properties": {"data_size": 29663680, "index_size": 1775872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26139456, "raw_average_key_size": 28, "raw_value_size": 108078300, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933552, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349124, "oldest_key_time": 0, "file_creation_time": 1732349124, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268392, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:28.166673 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268393.sst) /dev/shm/ToplingTemp/Topling-106742-Gg1n8x 2024/11/23-16:05:29.414512 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268393.sst): first pass time = 1.25's, 107.563'MB/sec 2024/11/23-16:05:30.972207 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268393.sst): old prealloc_size = 147639500, real_size = 30781408 2024/11/23-16:05:30.972780 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268393.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931989 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0288 GB avg-val = 116.01 avg-zval = 30.88 2024/11/23-16:05:30.973560 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268393.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:31.807678 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268393.sst): second pass time = 1.55's, 69.812'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.707'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3205.224'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 266.217'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35184.165'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.812'MB/sec zip pipeline throughput = 80.783'MB/sec entries = 931989 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.01 avg-zval = 30.88 usrkeys = 931989 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.58 avg-zval = 30.88 TagRS{ kinds = RS_Key0_TagN bytes = 145720 } TagArray{ size = 875013 bytes = 3062560 } seq expand size = 6125091 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.5132 value = 3.7575 dict = 1.00 all = 3.8519 } Zip/UnZip{ index = 0.0951 value = 0.2661 dict = 1.00 all = 0.2596 } ---------------------------- total value len = 9.421542 GB avg = 0.165 KB (by entry num) total key len = 1.571178 GB avg = 0.028 KB total ukey len = 1.090990 GB avg = 0.016 KB total ukey num = 0.070206057 Billion total entry num = 0.057013786 Billion write speed all = 0.425751113 MB/sec (with seq num) write speed all = 0.408085835 MB/sec (without seq num) 2024/11/23-16:05:31.850253 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268393: 931989 keys, 34844302 bytes, temperature: kWarm 2024/11/23-16:05:31.850321 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349131850285, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268393, "file_size": 34844302, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776192, "table_properties": {"data_size": 28775248, "index_size": 1772992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26095692, "raw_average_key_size": 28, "raw_value_size": 108122102, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931989, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349128, "oldest_key_time": 0, "file_creation_time": 1732349128, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268393, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:31.888039 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268394.sst) /dev/shm/ToplingTemp/Topling-106742-k67apx 2024/11/23-16:05:32.973982 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268394.sst): first pass time = 1.09's, 123.601'MB/sec 2024/11/23-16:05:34.552679 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268394.sst): old prealloc_size = 147639500, real_size = 34971376 2024/11/23-16:05:34.553316 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268394.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081570 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.10 avg-zval = 30.18 2024/11/23-16:05:34.553357 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268394.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:35.252498 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268394.sst): second pass time = 1.58's, 65.867'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 51.077'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3327.251'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.488'MB/sec write SST data time = 0.14's, 246.673'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.032'MB, throughput = 33222.337'MB/sec zip my value time = 1.58's, unzip length = 0.104'GB zip my value throughput = 65.867'MB/sec zip pipeline throughput = 67.855'MB/sec entries = 1081570 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.10 avg-zval = 30.18 usrkeys = 1081570 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.10 avg-zval = 30.18 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.03 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5162 value = 3.1839 dict = 1.00 all = 3.7550 } Zip/UnZip{ index = 0.0951 value = 0.3141 dict = 1.00 all = 0.2663 } ---------------------------- total value len = 9.525476 GB avg = 0.164 KB (by entry num) total key len = 1.601462 GB avg = 0.028 KB total ukey len = 1.112622 GB avg = 0.016 KB total ukey num = 0.071287627 Billion total entry num = 0.058095356 Billion write speed all = 0.430891917 MB/sec (with seq num) write speed all = 0.412893925 MB/sec (without seq num) 2024/11/23-16:05:35.298880 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268394: 1081570 keys, 35743793 bytes, temperature: kWarm 2024/11/23-16:05:35.298947 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349135298915, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268394, "file_size": 35743793, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32643808, "index_size": 2056960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30283960, "raw_average_key_size": 28, "raw_value_size": 103933849, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081570, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349131, "oldest_key_time": 0, "file_creation_time": 1732349131, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268394, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:35.337786 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268395.sst) /dev/shm/ToplingTemp/Topling-106742-27AM1x 2024/11/23-16:05:36.463959 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268395.sst): first pass time = 1.13's, 119.185'MB/sec 2024/11/23-16:05:38.075615 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268395.sst): old prealloc_size = 147639500, real_size = 34066752 2024/11/23-16:05:38.076248 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268395.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922010 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0321 GB avg-val = 117.57 avg-zval = 34.78 2024/11/23-16:05:38.076907 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268395.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:38.895920 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268395.sst): second pass time = 1.61's, 67.431'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 45.830'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2897.670'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 291.249'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35024.926'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.431'MB/sec zip pipeline throughput = 69.870'MB/sec entries = 922010 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.57 avg-zval = 34.78 usrkeys = 922010 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.67 avg-zval = 34.78 TagRS{ kinds = RS_Key0_TagN bytes = 144160 } TagArray{ size = 672153 bytes = 2520592 } seq expand size = 4705071 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4443 value = 3.3801 dict = 1.00 all = 3.5702 } Zip/UnZip{ index = 0.0957 value = 0.2958 dict = 1.00 all = 0.2801 } ---------------------------- total value len = 9.638582 GB avg = 0.163 KB (by entry num) total key len = 1.627279 GB avg = 0.028 KB total ukey len = 1.131062 GB avg = 0.016 KB total ukey num = 0.072209637 Billion total entry num = 0.059017366 Billion write speed all = 0.436210175 MB/sec (with seq num) write speed all = 0.417929122 MB/sec (without seq num) 2024/11/23-16:05:38.947080 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268395: 922010 keys, 37593644 bytes, temperature: kWarm 2024/11/23-16:05:38.947174 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349138947124, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268395, "file_size": 37593644, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775885, "table_properties": {"data_size": 32070512, "index_size": 1765568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25816280, "raw_average_key_size": 28, "raw_value_size": 108401508, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922010, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349135, "oldest_key_time": 0, "file_creation_time": 1732349135, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268395, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:38.997609 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268396.sst) /dev/shm/ToplingTemp/Topling-106742-Ls0J2w 2024/11/23-16:05:40.166741 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268396.sst): first pass time = 1.17's, 114.808'MB/sec 2024/11/23-16:05:41.963226 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268396.sst): old prealloc_size = 147639500, real_size = 32993392 2024/11/23-16:05:41.963768 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268396.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930785 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.20 avg-zval = 33.29 2024/11/23-16:05:41.965803 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268396.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:43.017386 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268396.sst): second pass time = 1.80's, 60.228'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.307'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3413.949'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.252'MB/sec write SST data time = 0.13's, 284.535'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34168.352'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.228'MB/sec zip pipeline throughput = 118.972'MB/sec entries = 930785 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.20 avg-zval = 33.29 usrkeys = 930785 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.20 avg-zval = 33.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930785 bytes = 3374112 } seq expand size = 6515495 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5136 value = 3.4900 dict = 1.00 all = 3.6062 } Zip/UnZip{ index = 0.0951 value = 0.2865 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 9.753254 GB avg = 0.163 KB (by entry num) total key len = 1.653341 GB avg = 0.028 KB total ukey len = 1.149678 GB avg = 0.016 KB total ukey num = 0.073140422 Billion total entry num = 0.059948151 Billion write speed all = 0.441588847 MB/sec (with seq num) write speed all = 0.423022440 MB/sec (without seq num) 2024/11/23-16:05:43.063147 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268396: 930785 keys, 37218738 bytes, temperature: kWarm 2024/11/23-16:05:43.063219 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349143063183, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268396, "file_size": 37218738, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775295, "table_properties": {"data_size": 30989856, "index_size": 1770624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26061980, "raw_average_key_size": 28, "raw_value_size": 108155774, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930785, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349138, "oldest_key_time": 0, "file_creation_time": 1732349138, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268396, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:43.100841 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268397.sst) /dev/shm/ToplingTemp/Topling-106742-oXZzTy 2024/11/23-16:05:43.942617 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268397.sst): first pass time = 0.84's, 159.454'MB/sec 2024/11/23-16:05:45.510568 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268397.sst): old prealloc_size = 147639500, real_size = 32775872 2024/11/23-16:05:45.511125 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268397.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932101 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 115.99 avg-zval = 33.01 2024/11/23-16:05:45.512549 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268397.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:46.395817 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268397.sst): second pass time = 1.57's, 68.989'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.672'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3310.254'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.13's, 273.537'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34010.035'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.989'MB/sec zip pipeline throughput = 70.879'MB/sec entries = 932101 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.99 avg-zval = 33.01 usrkeys = 932101 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.99 avg-zval = 33.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932101 bytes = 3262368 } seq expand size = 6524707 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5122 value = 3.5139 dict = 1.00 all = 3.6384 } Zip/UnZip{ index = 0.0951 value = 0.2846 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 9.867897 GB avg = 0.162 KB (by entry num) total key len = 1.679440 GB avg = 0.028 KB total ukey len = 1.168320 GB avg = 0.016 KB total ukey num = 0.074072523 Billion total entry num = 0.060880252 Billion write speed all = 0.446979016 MB/sec (with seq num) write speed all = 0.428126396 MB/sec (without seq num) 2024/11/23-16:05:46.442835 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268397: 932101 keys, 36888768 bytes, temperature: kWarm 2024/11/23-16:05:46.442903 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349146442867, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268397, "file_size": 36888768, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 30769264, "index_size": 1773376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26098828, "raw_average_key_size": 28, "raw_value_size": 108118950, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932101, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349143, "oldest_key_time": 0, "file_creation_time": 1732349143, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268397, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:46.482132 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268398.sst) /dev/shm/ToplingTemp/Topling-106742-kRQFov 2024/11/23-16:05:47.381357 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268398.sst): first pass time = 0.90's, 149.266'MB/sec 2024/11/23-16:05:48.989787 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268398.sst): old prealloc_size = 147639500, real_size = 33093584 2024/11/23-16:05:48.990371 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268398.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930670 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.22 avg-zval = 33.41 2024/11/23-16:05:48.992990 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268398.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:50.022425 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268398.sst): second pass time = 1.61's, 67.301'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 36.252'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3152.746'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.13's, 284.129'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32232.610'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.301'MB/sec zip pipeline throughput = 67.301'MB/sec entries = 930670 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 33.41 usrkeys = 930670 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 33.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930670 bytes = 3490032 } seq expand size = 6514690 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5139 value = 3.4789 dict = 1.00 all = 3.5854 } Zip/UnZip{ index = 0.0951 value = 0.2875 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 9.982571 GB avg = 0.162 KB (by entry num) total key len = 1.705498 GB avg = 0.028 KB total ukey len = 1.186933 GB avg = 0.016 KB total ukey num = 0.075003193 Billion total entry num = 0.061810922 Billion write speed all = 0.452363045 MB/sec (with seq num) write speed all = 0.433224913 MB/sec (without seq num) 2024/11/23-16:05:50.070446 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268398: 930670 keys, 37434546 bytes, temperature: kWarm 2024/11/23-16:05:50.070503 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349150070469, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268398, "file_size": 37434546, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012774578, "table_properties": {"data_size": 31090368, "index_size": 1770368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058760, "raw_average_key_size": 28, "raw_value_size": 108159060, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930670, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349146, "oldest_key_time": 0, "file_creation_time": 1732349146, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268398, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:50.110198 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268399.sst) /dev/shm/ToplingTemp/Topling-106742-BBX92u 2024/11/23-16:05:50.965213 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268399.sst): first pass time = 0.85's, 156.983'MB/sec 2024/11/23-16:05:52.303771 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268399.sst): old prealloc_size = 147639500, real_size = 34341040 2024/11/23-16:05:52.304334 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268399.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928829 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.50 avg-zval = 34.81 2024/11/23-16:05:52.306948 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268399.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:53.155706 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268399.sst): second pass time = 1.34's, 80.883'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.264'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3294.530'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 294.234'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33095.791'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.883'MB/sec zip pipeline throughput = 80.883'MB/sec entries = 928829 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 34.81 usrkeys = 928829 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 34.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928829 bytes = 3367024 } seq expand size = 6501803 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4451 value = 3.3471 dict = 1.00 all = 3.4806 } Zip/UnZip{ index = 0.0957 value = 0.2988 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 10.097284 GB avg = 0.161 KB (by entry num) total key len = 1.731506 GB avg = 0.028 KB total ukey len = 1.205510 GB avg = 0.016 KB total ukey num = 0.075932022 Billion total entry num = 0.062739751 Billion write speed all = 0.457753802 MB/sec (with seq num) write speed all = 0.438330438 MB/sec (without seq num) 2024/11/23-16:05:53.202441 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268399: 928829 keys, 38561364 bytes, temperature: kWarm 2024/11/23-16:05:53.202512 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349153202475, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268399, "file_size": 38561364, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 32330144, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007212, "raw_average_key_size": 28, "raw_value_size": 108210614, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349150, "oldest_key_time": 0, "file_creation_time": 1732349150, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268399, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:53.243500 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268400.sst) /dev/shm/ToplingTemp/Topling-106742-zl4HHx 2024/11/23-16:05:54.265128 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268400.sst): first pass time = 1.02's, 131.382'MB/sec 2024/11/23-16:05:55.599515 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268400.sst): old prealloc_size = 147639500, real_size = 33910400 2024/11/23-16:05:55.600090 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268400.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925888 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0319 GB avg-val = 116.96 avg-zval = 34.47 2024/11/23-16:05:55.602733 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268400.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:56.446276 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268400.sst): second pass time = 1.33's, 81.203'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.022'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3205.434'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 288.861'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33524.244'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.203'MB/sec zip pipeline throughput = 81.203'MB/sec entries = 925888 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 34.47 usrkeys = 925888 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 34.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925888 bytes = 3356352 } seq expand size = 6481216 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5146 value = 3.3929 dict = 1.00 all = 3.5208 } Zip/UnZip{ index = 0.0951 value = 0.2947 dict = 1.00 all = 0.2840 } ---------------------------- total value len = 10.212058 GB avg = 0.160 KB (by entry num) total key len = 1.757430 GB avg = 0.028 KB total ukey len = 1.224028 GB avg = 0.016 KB total ukey num = 0.076857910 Billion total entry num = 0.063665639 Billion write speed all = 0.463139636 MB/sec (with seq num) write speed all = 0.443432139 MB/sec (without seq num) 2024/11/23-16:05:56.493034 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268400: 925888 keys, 38121822 bytes, temperature: kWarm 2024/11/23-16:05:56.493100 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349156493067, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268400, "file_size": 38121822, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012776021, "table_properties": {"data_size": 31917616, "index_size": 1761152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924864, "raw_average_key_size": 28, "raw_value_size": 108292916, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925888, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349153, "oldest_key_time": 0, "file_creation_time": 1732349153, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268400, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:56.531704 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268401.sst) /dev/shm/ToplingTemp/Topling-106742-479mwx 2024/11/23-16:05:57.581643 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268401.sst): first pass time = 1.05's, 127.839'MB/sec 2024/11/23-16:05:58.944110 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268401.sst): old prealloc_size = 147639500, real_size = 36164592 2024/11/23-16:05:58.944657 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268401.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925584 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0342 GB avg-val = 117.01 avg-zval = 36.91 2024/11/23-16:05:58.945390 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268401.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:05:59.759029 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268401.sst): second pass time = 1.36's, 79.568'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 49.029'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3380.648'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 298.284'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36707.941'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.568'MB/sec zip pipeline throughput = 79.568'MB/sec entries = 925584 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.01 avg-zval = 36.91 usrkeys = 925584 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.62 avg-zval = 36.91 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 742587 bytes = 2784720 } seq expand size = 5198109 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4451 value = 3.1703 dict = 1.00 all = 3.3595 } Zip/UnZip{ index = 0.0957 value = 0.3154 dict = 1.00 all = 0.2977 } ---------------------------- total value len = 10.325557 GB avg = 0.160 KB (by entry num) total key len = 1.783347 GB avg = 0.028 KB total ukey len = 1.242539 GB avg = 0.016 KB total ukey num = 0.077783494 Billion total entry num = 0.064591223 Billion write speed all = 0.468474055 MB/sec (with seq num) write speed all = 0.448482609 MB/sec (without seq num) 2024/11/23-16:05:59.809747 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268401: 925584 keys, 39952294 bytes, temperature: kWarm 2024/11/23-16:05:59.809818 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349159809781, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268401, "file_size": 39952294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 34160736, "index_size": 1772288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25916352, "raw_average_key_size": 28, "raw_value_size": 108301428, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925584, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349156, "oldest_key_time": 0, "file_creation_time": 1732349156, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268401, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:05:59.848972 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268402.sst) /dev/shm/ToplingTemp/Topling-106742-dy1QCw 2024/11/23-16:06:00.862510 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268402.sst): first pass time = 1.01's, 132.437'MB/sec 2024/11/23-16:06:02.205620 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268402.sst): old prealloc_size = 147639500, real_size = 32776992 2024/11/23-16:06:02.206229 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268402.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931641 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.07 avg-zval = 33.02 2024/11/23-16:06:02.208308 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268402.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:06:03.076603 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268402.sst): second pass time = 1.34's, 80.554'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.487'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3052.939'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 281.631'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33008.300'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.554'MB/sec zip pipeline throughput = 80.554'MB/sec entries = 931641 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.07 avg-zval = 33.02 usrkeys = 931641 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.07 avg-zval = 33.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931641 bytes = 3377216 } seq expand size = 6521487 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4455 value = 3.5153 dict = 1.00 all = 3.6270 } Zip/UnZip{ index = 0.0957 value = 0.2845 dict = 1.00 all = 0.2757 } ---------------------------- total value len = 10.440211 GB avg = 0.159 KB (by entry num) total key len = 1.809433 GB avg = 0.028 KB total ukey len = 1.261172 GB avg = 0.016 KB total ukey num = 0.078715135 Billion total entry num = 0.065522864 Billion write speed all = 0.473858213 MB/sec (with seq num) write speed all = 0.453581020 MB/sec (without seq num) 2024/11/23-16:06:03.128157 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268402: 931641 keys, 37004874 bytes, temperature: kWarm 2024/11/23-16:06:03.128226 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349163128190, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268402, "file_size": 37004874, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 30760080, "index_size": 1783808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26085948, "raw_average_key_size": 28, "raw_value_size": 108131910, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931641, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349159, "oldest_key_time": 0, "file_creation_time": 1732349159, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268402, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:06:03.175121 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268403.sst) /dev/shm/ToplingTemp/Topling-106742-VUcIYw 2024/11/23-16:06:04.073030 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268403.sst): first pass time = 0.90's, 149.484'MB/sec 2024/11/23-16:06:05.375136 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268403.sst): old prealloc_size = 147639500, real_size = 31942096 2024/11/23-16:06:05.375724 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268403.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936596 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0299 GB avg-val = 115.30 avg-zval = 31.95 2024/11/23-16:06:05.377835 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268403.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:06:06.239644 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268403.sst): second pass time = 1.30's, 83.051'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.858'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.639'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 274.670'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 30886.320'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.051'MB/sec zip pipeline throughput = 83.051'MB/sec entries = 936596 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.30 avg-zval = 31.95 usrkeys = 936596 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.30 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936596 bytes = 3395168 } seq expand size = 6556172 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5135 value = 3.6087 dict = 1.00 all = 3.7091 } Zip/UnZip{ index = 0.0951 value = 0.2771 dict = 1.00 all = 0.2696 } ---------------------------- total value len = 10.554760 GB avg = 0.159 KB (by entry num) total key len = 1.835657 GB avg = 0.028 KB total ukey len = 1.279904 GB avg = 0.016 KB total ukey num = 0.079651731 Billion total entry num = 0.066459460 Billion write speed all = 0.479245195 MB/sec (with seq num) write speed all = 0.458680672 MB/sec (without seq num) 2024/11/23-16:06:06.289074 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268403: 936596 keys, 36185736 bytes, temperature: kWarm 2024/11/23-16:06:06.289143 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349166289107, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268403, "file_size": 36185736, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 29926080, "index_size": 1781696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26224688, "raw_average_key_size": 28, "raw_value_size": 107993132, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936596, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349163, "oldest_key_time": 0, "file_creation_time": 1732349163, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268403, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:06:06.330510 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268404.sst) /dev/shm/ToplingTemp/Topling-106742-ja5Xcz 2024/11/23-16:06:07.366785 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268404.sst): first pass time = 1.04's, 129.525'MB/sec 2024/11/23-16:06:08.706368 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268404.sst): old prealloc_size = 147639500, real_size = 35270464 2024/11/23-16:06:08.706935 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268404.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926500 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 116.87 avg-zval = 35.90 2024/11/23-16:06:08.709052 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268404.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:06:09.546969 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268404.sst): second pass time = 1.34's, 80.878'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.109'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.729'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 303.384'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35532.217'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.878'MB/sec zip pipeline throughput = 80.878'MB/sec entries = 926500 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.87 avg-zval = 35.90 usrkeys = 926500 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.87 avg-zval = 35.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926500 bytes = 3474384 } seq expand size = 6485500 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4433 value = 3.2550 dict = 1.00 all = 3.3894 } Zip/UnZip{ index = 0.0958 value = 0.3072 dict = 1.00 all = 0.2950 } ---------------------------- total value len = 10.669521 GB avg = 0.158 KB (by entry num) total key len = 1.861599 GB avg = 0.028 KB total ukey len = 1.298434 GB avg = 0.016 KB total ukey num = 0.080578231 Billion total entry num = 0.067385960 Billion write speed all = 0.484625418 MB/sec (with seq num) write speed all = 0.463776876 MB/sec (without seq num) 2024/11/23-16:06:09.601564 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268404: 926500 keys, 39599342 bytes, temperature: kWarm 2024/11/23-16:06:09.601635 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349169601599, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268404, "file_size": 39599342, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33264304, "index_size": 1774336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942000, "raw_average_key_size": 28, "raw_value_size": 108275736, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926500, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349166, "oldest_key_time": 0, "file_creation_time": 1732349166, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268404, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:06:09.639589 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268405.sst) /dev/shm/ToplingTemp/Topling-106742-YY2GLu 2024/11/23-16:06:10.664577 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268405.sst): first pass time = 1.02's, 130.951'MB/sec 2024/11/23-16:06:11.970384 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268405.sst): old prealloc_size = 147639500, real_size = 31832720 2024/11/23-16:06:11.970962 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268405.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0298 GB avg-val = 116.13 avg-zval = 32.03 2024/11/23-16:06:11.973463 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268405.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:06:12.811443 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268405.sst): second pass time = 1.31's, 82.867'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.875'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3115.142'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 279.821'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35072.741'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.867'MB/sec zip pipeline throughput = 82.867'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 32.03 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 32.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5123 value = 3.6255 dict = 1.00 all = 3.7221 } Zip/UnZip{ index = 0.0951 value = 0.2758 dict = 1.00 all = 0.2687 } ---------------------------- total value len = 10.784183 GB avg = 0.158 KB (by entry num) total key len = 1.887674 GB avg = 0.028 KB total ukey len = 1.317059 GB avg = 0.016 KB total ukey num = 0.081509473 Billion total entry num = 0.068317202 Billion write speed all = 0.490006361 MB/sec (with seq num) write speed all = 0.468872370 MB/sec (without seq num) 2024/11/23-16:06:12.859307 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268405: 931242 keys, 36060112 bytes, temperature: kWarm 2024/11/23-16:06:12.859386 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349172859341, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268405, "file_size": 36060112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 29828032, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349169, "oldest_key_time": 0, "file_creation_time": 1732349169, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268405, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:06:12.900295 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268406.sst) /dev/shm/ToplingTemp/Topling-106742-S65BMv 2024/11/23-16:06:13.930907 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268406.sst): first pass time = 1.03's, 130.235'MB/sec 2024/11/23-16:06:15.280611 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268406.sst): old prealloc_size = 147639500, real_size = 35409264 2024/11/23-16:06:15.281179 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268406.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0334 GB avg-val = 117.46 avg-zval = 36.22 2024/11/23-16:06:15.282011 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268406.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:06:16.122639 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268406.sst): second pass time = 1.35's, 80.377'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.182'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3219.786'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 310.385'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36261.725'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.377'MB/sec zip pipeline throughput = 80.377'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 36.22 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 36.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5127 value = 3.2427 dict = 1.00 all = 3.3784 } Zip/UnZip{ index = 0.0951 value = 0.3084 dict = 1.00 all = 0.2960 } ---------------------------- total value len = 10.899024 GB avg = 0.157 KB (by entry num) total key len = 1.913510 GB avg = 0.028 KB total ukey len = 1.335513 GB avg = 0.016 KB total ukey num = 0.082432170 Billion total entry num = 0.069239899 Billion write speed all = 0.495382740 MB/sec (with seq num) write speed all = 0.473966054 MB/sec (without seq num) 2024/11/23-16:06:16.173504 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268406: 922697 keys, 39727734 bytes, temperature: kWarm 2024/11/23-16:06:16.173576 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349176173539, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268406, "file_size": 39727734, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33423008, "index_size": 1755392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349172, "oldest_key_time": 0, "file_creation_time": 1732349172, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268406, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:06:16.212300 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268407.sst) /dev/shm/ToplingTemp/Topling-106742-5bVpRy 2024/11/23-16:06:17.241307 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268407.sst): first pass time = 1.03's, 130.440'MB/sec 2024/11/23-16:06:18.558360 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268407.sst): old prealloc_size = 147639500, real_size = 32854624 2024/11/23-16:06:18.558914 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268407.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.15 avg-zval = 33.12 2024/11/23-16:06:18.559756 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268407.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:06:19.439153 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268407.sst): second pass time = 1.32's, 82.208'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.233'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3364.429'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 290.785'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35000.974'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.208'MB/sec zip pipeline throughput = 82.208'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 33.12 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 33.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4441 value = 3.5069 dict = 1.00 all = 3.6082 } Zip/UnZip{ index = 0.0957 value = 0.2852 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 11.013689 GB avg = 0.157 KB (by entry num) total key len = 1.939580 GB avg = 0.028 KB total ukey len = 1.354134 GB avg = 0.016 KB total ukey num = 0.083363253 Billion total entry num = 0.070170982 Billion write speed all = 0.500759911 MB/sec (with seq num) write speed all = 0.479058013 MB/sec (without seq num) 2024/11/23-16:06:19.488528 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268407: 931083 keys, 37197768 bytes, temperature: kWarm 2024/11/23-16:06:19.488604 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349179488566, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268407, "file_size": 37197768, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30838672, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349176, "oldest_key_time": 0, "file_creation_time": 1732349176, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268407, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:06:19.532727 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268408.sst) /dev/shm/ToplingTemp/Topling-106742-qi8XQv 2024/11/23-16:06:20.550991 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268408.sst): first pass time = 1.02's, 131.818'MB/sec 2024/11/23-16:06:21.910528 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268408.sst): old prealloc_size = 147639500, real_size = 33561456 2024/11/23-16:06:21.911122 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268408.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927218 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.75 avg-zval = 34.03 2024/11/23-16:06:21.911745 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268408.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:06:22.724242 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268408.sst): second pass time = 1.36's, 79.701'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 45.365'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3109.304'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 282.679'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32544.546'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.701'MB/sec zip pipeline throughput = 79.701'MB/sec entries = 927218 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.75 avg-zval = 34.03 usrkeys = 927218 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.64 avg-zval = 34.03 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 647517 bytes = 2347264 } seq expand size = 4532619 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4435 value = 3.4308 dict = 1.00 all = 3.6361 } Zip/UnZip{ index = 0.0958 value = 0.2915 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 11.126477 GB avg = 0.156 KB (by entry num) total key len = 1.965542 GB avg = 0.028 KB total ukey len = 1.372679 GB avg = 0.016 KB total ukey num = 0.084290471 Billion total entry num = 0.071098200 Billion write speed all = 0.506059590 MB/sec (with seq num) write speed all = 0.484073722 MB/sec (without seq num) 2024/11/23-16:06:22.772661 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268408: 927218 keys, 36912880 bytes, temperature: kWarm 2024/11/23-16:06:22.772738 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349182772695, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268408, "file_size": 36912880, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 31553760, "index_size": 1775680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962104, "raw_average_key_size": 28, "raw_value_size": 108255724, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927218, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349179, "oldest_key_time": 0, "file_creation_time": 1732349179, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268408, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:06:22.818254 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268409.sst) /dev/shm/ToplingTemp/Topling-106742-CMWUAv 2024/11/23-16:06:23.635760 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268409.sst): first pass time = 0.82's, 164.188'MB/sec 2024/11/23-16:06:25.145941 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268409.sst): old prealloc_size = 147639500, real_size = 36875840 2024/11/23-16:06:25.146537 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268409.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926671 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.84 avg-zval = 37.62 2024/11/23-16:06:25.146866 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268409.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:06:25.866636 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268409.sst): second pass time = 1.51's, 71.731'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 53.230'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3213.328'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.391'MB/sec write SST data time = 0.13's, 302.376'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35866.994'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.731'MB/sec zip pipeline throughput = 71.731'MB/sec entries = 926671 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.84 avg-zval = 37.62 usrkeys = 926671 avg-key = 20.00 avg-zkey = 1.93 avg-val = 118.01 avg-zval = 37.62 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.3768 value = 3.1061 dict = 1.00 all = 3.4988 } Zip/UnZip{ index = 0.0964 value = 0.3220 dict = 1.00 all = 0.2858 } ---------------------------- total value len = 11.235829 GB avg = 0.156 KB (by entry num) total key len = 1.991489 GB avg = 0.028 KB total ukey len = 1.391212 GB avg = 0.016 KB total ukey num = 0.085217142 Billion total entry num = 0.072024871 Billion write speed all = 0.511227334 MB/sec (with seq num) write speed all = 0.488957615 MB/sec (without seq num) 2024/11/23-16:06:25.915243 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268409: 926671 keys, 38361509 bytes, temperature: kWarm 2024/11/23-16:06:25.915314 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349185915279, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268409, "file_size": 38361509, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34857968, "index_size": 1786048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946788, "raw_average_key_size": 28, "raw_value_size": 108271074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926671, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349182, "oldest_key_time": 0, "file_creation_time": 1732349182, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268409, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:06:25.954574 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268410.sst) /dev/shm/ToplingTemp/Topling-106742-R5emgw 2024/11/23-16:06:26.816585 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268410.sst): first pass time = 0.86's, 155.710'MB/sec 2024/11/23-16:06:28.172267 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268410.sst): old prealloc_size = 147639500, real_size = 35477232 2024/11/23-16:06:28.172858 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268410.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924891 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 117.12 avg-zval = 36.21 2024/11/23-16:06:28.173477 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268410.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-16:06:28.979075 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268410.sst): second pass time = 1.35's, 79.947'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 48.043'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3116.391'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.343'MB/sec write SST data time = 0.13's, 295.109'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35742.552'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.947'MB/sec zip pipeline throughput = 79.947'MB/sec entries = 924891 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 36.21 usrkeys = 924891 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.23 avg-zval = 36.21 TagRS{ kinds = RS_Key0_TagN bytes = 144600 } TagArray{ size = 675272 bytes = 2279056 } seq expand size = 4726904 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5128 value = 3.2348 dict = 1.00 all = 3.4628 } Zip/UnZip{ index = 0.0951 value = 0.3091 dict = 1.00 all = 0.2888 } ---------------------------- total value len = 11.348877 GB avg = 0.156 KB (by entry num) total key len = 2.017386 GB avg = 0.028 KB total ukey len = 1.409710 GB avg = 0.016 KB total ukey num = 0.086142033 Billion total entry num = 0.072949762 Billion write speed all = 0.516535324 MB/sec (with seq num) write speed all = 0.493982346 MB/sec (without seq num) 2024/11/23-16:06:29.029625 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268410: 924891 keys, 38760274 bytes, temperature: kWarm 2024/11/23-16:06:29.029695 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349189029658, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268410, "file_size": 38760274, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4094023557, "table_properties": {"data_size": 33486304, "index_size": 1759552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25896948, "raw_average_key_size": 28, "raw_value_size": 108320922, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924891, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349185, "oldest_key_time": 0, "file_creation_time": 1732349185, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268410, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:06:29.066945 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268411.sst) /dev/shm/ToplingTemp/Topling-106742-UYcdGw 2024/11/23-16:06:29.188879 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268411.sst): first pass time = 0.12's, 225.698'MB/sec 2024/11/23-16:06:29.443863 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268411.sst): old prealloc_size = 147639500, real_size = 8050800 2024/11/23-16:06:29.444049 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268411.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 189950 prefix = 4 raw-key = 0.0030 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0222 GB zip-val = 0.0076 GB avg-val = 116.80 avg-zval = 40.22 2024/11/23-16:06:29.444471 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268411.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.028 GB 2024/11/23-16:06:29.611608 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00/268411.sst): second pass time = 0.25's, 87.337'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 52.656'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2197.125'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.338'MB/sec write SST data time = 0.03's, 325.253'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.233'MB, throughput = 8439.171'MB/sec zip my value time = 0.25's, unzip length = 0.022'GB zip my value throughput = 87.337'MB/sec zip pipeline throughput = 87.337'MB/sec entries = 189950 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.80 avg-zval = 40.22 usrkeys = 189950 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.80 avg-zval = 40.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 189950 bytes = 593616 } seq expand size = 1329650 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0038 GB value = 0.0222 GB dict = 0.23 MB all = 0.0275 GB } __ZipSize{ index = 0.0004 GB value = 0.0076 GB dict = 0.23 MB all = 0.0088 GB } UnZip/Zip{ index = 10.4727 value = 2.9038 dict = 1.00 all = 3.1143 } Zip/UnZip{ index = 0.0955 value = 0.3444 dict = 1.00 all = 0.3211 } ---------------------------- total value len = 11.372392 GB avg = 0.155 KB (by entry num) total key len = 2.022704 GB avg = 0.028 KB total ukey len = 1.413509 GB avg = 0.016 KB total ukey num = 0.086331983 Billion total entry num = 0.073139712 Billion write speed all = 0.517636952 MB/sec (with seq num) write speed all = 0.495025802 MB/sec (without seq num) 2024/11/23-16:06:29.629199 139957659977280 [compaction_job.cc:1948] [default] [JOB 30146] Generated table #268411: 189950 keys, 8831647 bytes, temperature: kWarm 2024/11/23-16:06:29.629260 139957659977280 EVENT_LOG_v1 {"time_micros": 1732349189629228, "cf_name": "default", "job": 30146, "event": "table_file_creation", "file_number": 268411, "file_size": 8831647, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4072856642, "largest_seqno": 4094023576, "table_properties": {"data_size": 7640352, "index_size": 362752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5318600, "raw_average_key_size": 28, "raw_value_size": 22185744, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 189950, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732349189, "oldest_key_time": 0, "file_creation_time": 1732349189, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268411, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-16:06:29.642486 139957659977280 [compaction_job.cc:703] job-30146: subcompact[0], size: 0.840203 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-30146/att-00 [268389,268390,268391,268392,268393,268394,268395,268396,268397,268398,268399,268400,268401,268402,268403,268404,268405,268406,268407,268408,268409,268410,268411] 2024/11/23-16:06:29.775621 139957659977280 (Original Log Time 2024/11/23-16:05:11.196220) EVENT_LOG_v1 {"time_micros": 1732349111196178, "job": 30146, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [268374], "files_L3": [268342, 268343, 268344, 268309, 268310, 268311, 268312, 268313, 268314, 268315, 268316, 268317, 268318, 268319, 268320, 268321, 268322, 268323, 268324, 268325, 268326, 268327, 268328], "score": 0.992698, "input_data_size": 861764857, "oldest_snapshot_seqno": 4094137791} 2024/11/23-16:06:29.775624 139957659977280 (Original Log Time 2024/11/23-16:06:29.773995) [compaction_job.cc:2022] [default] [JOB 30146] Compacted 1@2 + 23@3 files to L3 => 840202965 bytes 2024/11/23-16:06:29.775626 139957659977280 (Original Log Time 2024/11/23-16:06:29.775550) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.0 rd, 10.7 wr, level 3, files in(1, 23) out(23 +0 blob) MB in(25.7, 796.2 +0.0 blob) out(801.3 +0.0 blob), read-write-amplify(63.2) write-amplify(31.2) OK, records in: 20886700, records dropped: 105152 output_compression: Snappy 2024/11/23-16:06:29.775629 139957659977280 (Original Log Time 2024/11/23-16:06:29.775596) EVENT_LOG_v1 {"time_micros": 1732349189775572, "cf": "default", "job": 30146, "event": "compaction_finished", "compaction_time_micros": 78446271, "compaction_time_cpu_micros": 33190989, "output_level": 3, "num_output_files": 23, "total_output_size": 840202965, "num_input_records": 20886700, "num_output_records": 20781548, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 629488838, "file_range_sync_nanos": 0, "file_fsync_nanos": 16613, "file_prepare_write_nanos": 63235, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-16:06:29.901392 139957659977280 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732349111,"executesMs":78737,"compactionJobId":30146,"attempt":0,"compactionInputRawBytes":2995227717,"compactionInputZipBytes":861764857,"compactionOutputRawBytes":0,"compactionOutputZipBytes":840202965,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-e127af64d064d5715782fbb02b354bf1-04fc4dfc2752ec5f-00"}}, response = 2024/11/23-16:06:29.901482 139957659977280 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-30146/att-00: olev 3, work 78.737 s, result 7.732 ms, install 1.650 ms, input{raw 2.790 GiB zip 821.843 MiB}