2024/11/23-14:17:06.963818 139957542479424 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00: fork to child time = 0.007628 sec 2024/11/23-14:17:06.964601 139957542479424 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00 2024/11/23-14:17:06.971923 139957542479424 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00 2024/11/23-14:17:06.972879 139957542479424 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:17:06.973161 139957542479424 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:17:07.026416 139957542479424 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27488/att-00/MANIFEST-206347 2024/11/23-14:17:07.067035 139957542479424 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27488/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 262687, last_sequence is 4014770573, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 262481 2024/11/23-14:17:07.067042 139957542479424 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:17:07.067167 139957542479424 [version_set.cc:5721] Creating manifest 262687 2024/11/23-14:17:07.099775 139957542479424 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:17:07.122349 139957542479424 [compaction_job.cc:2437] [default] [JOB 27488] Compacting 1@2 + 25@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-14:17:07.122368 139957542479424 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20582 Base level 2, inputs: [262619(27MB)], [262684(35MB) 262685(7754KB) 262588(35MB) 262589(36MB) 262590(33MB) 262591(21MB) 262566(34MB) 262567(35MB) 262568(33MB) 262569(35MB) 262570(34MB) 262571(36MB) 262572(32MB) 262573(34MB) 262574(37MB) 262575(32MB) 262576(33MB) 262577(33MB) 262578(3288KB) 206445(35MB) 206449(7402KB) 204580(4176KB) 204186(4215KB) 203758(3907KB) 203310(2325KB)] 2024/11/23-14:17:07.132408 139957542479424 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:17:07.132434 139957542479424 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:17:07.132450 139957542479424 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:17:07.132454 139957542479424 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:17:07.132465 139957542479424 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:17:07.861958 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262688.sst) /dev/shm/ToplingTemp/Topling-69818-dVdluu 2024/11/23-14:17:08.408526 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262688.sst): first pass time = 0.55's, 245.594'MB/sec 2024/11/23-14:17:09.712240 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262688.sst): old prealloc_size = 147639500, real_size = 32711296 2024/11/23-14:17:09.712810 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262688.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.12 avg-zval = 32.97 2024/11/23-14:17:09.713640 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262688.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:10.564364 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262688.sst): second pass time = 1.30's, 83.033'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.214'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3241.142'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.12's, 294.663'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35136.639'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.033'MB/sec zip pipeline throughput = 83.033'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 32.97 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 32.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5116 value = 3.5218 dict = 1.00 all = 3.6450 } Zip/UnZip{ index = 0.0951 value = 0.2839 dict = 1.00 all = 0.2743 } ---------------------------- total value len = 8.963280 GB avg = 0.168 KB (by entry num) total key len = 1.466898 GB avg = 0.028 KB total ukey len = 1.016504 GB avg = 0.015 KB total ukey num = 0.066481744 Billion total entry num = 0.053289473 Billion write speed all = 0.539910396 MB/sec (with seq num) write speed all = 0.517842477 MB/sec (without seq num) 2024/11/23-14:17:10.611881 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262688: 931309 keys, 36822372 bytes, temperature: kWarm 2024/11/23-14:17:10.611996 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342630611921, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262688, "file_size": 36822372, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 30706288, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342627, "oldest_key_time": 0, "file_creation_time": 1732342627, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262688, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:10.653119 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262689.sst) /dev/shm/ToplingTemp/Topling-69818-LcV4mv 2024/11/23-14:17:11.788992 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262689.sst): first pass time = 1.14's, 118.167'MB/sec 2024/11/23-14:17:13.105866 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262689.sst): old prealloc_size = 147639500, real_size = 32855824 2024/11/23-14:17:13.106469 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262689.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.09 avg-zval = 33.12 2024/11/23-14:17:13.107318 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262689.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:13.956157 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262689.sst): second pass time = 1.32's, 82.192'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.751'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3046.739'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 296.078'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35788.104'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.192'MB/sec zip pipeline throughput = 82.192'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 33.12 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 33.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5122 value = 3.5052 dict = 1.00 all = 3.6080 } Zip/UnZip{ index = 0.0951 value = 0.2853 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 9.077937 GB avg = 0.167 KB (by entry num) total key len = 1.492979 GB avg = 0.028 KB total ukey len = 1.035133 GB avg = 0.015 KB total ukey num = 0.067413206 Billion total entry num = 0.054220935 Billion write speed all = 0.547099539 MB/sec (with seq num) write speed all = 0.524649830 MB/sec (without seq num) 2024/11/23-14:17:14.003535 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262689: 931462 keys, 37200032 bytes, temperature: kWarm 2024/11/23-14:17:14.003601 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342634003564, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262689, "file_size": 37200032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 30850624, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342630, "oldest_key_time": 0, "file_creation_time": 1732342630, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262689, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:14.040437 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262690.sst) /dev/shm/ToplingTemp/Topling-69818-xOl79v 2024/11/23-14:17:14.888247 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262690.sst): first pass time = 0.85's, 158.319'MB/sec 2024/11/23-14:17:16.234788 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262690.sst): old prealloc_size = 147639500, real_size = 34345664 2024/11/23-14:17:16.235337 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262690.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 117.04 avg-zval = 34.96 2024/11/23-14:17:16.236147 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262690.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:17.076680 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262690.sst): second pass time = 1.35's, 80.508'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.798'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3256.075'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 304.981'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37211.142'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.508'MB/sec zip pipeline throughput = 80.508'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 34.96 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 34.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5137 value = 3.3476 dict = 1.00 all = 3.4812 } Zip/UnZip{ index = 0.0951 value = 0.2987 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 9.192721 GB avg = 0.167 KB (by entry num) total key len = 1.518890 GB avg = 0.028 KB total ukey len = 1.053642 GB avg = 0.015 KB total ukey num = 0.068338611 Billion total entry num = 0.055146340 Billion write speed all = 0.554291743 MB/sec (with seq num) write speed all = 0.531462565 MB/sec (without seq num) 2024/11/23-14:17:17.123654 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262690: 925405 keys, 38555454 bytes, temperature: kWarm 2024/11/23-14:17:17.123723 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342637123687, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262690, "file_size": 38555454, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 32353776, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342634, "oldest_key_time": 0, "file_creation_time": 1732342634, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262690, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:17.158215 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262691.sst) /dev/shm/ToplingTemp/Topling-69818-tGlJsv 2024/11/23-14:17:18.085034 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262691.sst): first pass time = 0.93's, 144.821'MB/sec 2024/11/23-14:17:19.375864 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262691.sst): old prealloc_size = 147639500, real_size = 32604224 2024/11/23-14:17:19.376482 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262691.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925332 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 117.05 avg-zval = 33.06 2024/11/23-14:17:19.377267 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262691.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:20.209145 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262691.sst): second pass time = 1.29's, 83.991'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.180'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3006.183'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 289.953'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36167.799'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.991'MB/sec zip pipeline throughput = 83.991'MB/sec entries = 925332 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.05 avg-zval = 33.06 usrkeys = 925332 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.05 avg-zval = 33.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925332 bytes = 3354336 } seq expand size = 6477324 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.3770 value = 3.5407 dict = 1.00 all = 3.6459 } Zip/UnZip{ index = 0.0964 value = 0.2824 dict = 1.00 all = 0.2743 } ---------------------------- total value len = 9.307507 GB avg = 0.166 KB (by entry num) total key len = 1.544799 GB avg = 0.028 KB total ukey len = 1.072148 GB avg = 0.015 KB total ukey num = 0.069263943 Billion total entry num = 0.056071672 Billion write speed all = 0.561481253 MB/sec (with seq num) write speed all = 0.538272772 MB/sec (without seq num) 2024/11/23-14:17:20.253998 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262691: 925332 keys, 36813368 bytes, temperature: kWarm 2024/11/23-14:17:20.254068 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342640254031, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262691, "file_size": 36813368, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775987, "table_properties": {"data_size": 30589296, "index_size": 1783424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25909296, "raw_average_key_size": 28, "raw_value_size": 108308480, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925332, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342637, "oldest_key_time": 0, "file_creation_time": 1732342637, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262691, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:20.291024 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262692.sst) /dev/shm/ToplingTemp/Topling-69818-ig0mGt 2024/11/23-14:17:21.607680 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262692.sst): first pass time = 1.32's, 101.941'MB/sec 2024/11/23-14:17:22.922513 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262692.sst): old prealloc_size = 147639500, real_size = 33170080 2024/11/23-14:17:22.923122 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262692.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928714 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.52 avg-zval = 33.54 2024/11/23-14:17:22.923915 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262692.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:23.783138 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262692.sst): second pass time = 1.31's, 82.355'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.446'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3056.392'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 287.967'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35053.780'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.355'MB/sec zip pipeline throughput = 82.355'MB/sec entries = 928714 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.52 avg-zval = 33.54 usrkeys = 928714 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.52 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928714 bytes = 3366608 } seq expand size = 6500998 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3762 value = 3.4742 dict = 1.00 all = 3.5897 } Zip/UnZip{ index = 0.0964 value = 0.2878 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 9.422222 GB avg = 0.165 KB (by entry num) total key len = 1.570803 GB avg = 0.028 KB total ukey len = 1.090722 GB avg = 0.016 KB total ukey num = 0.070192657 Billion total entry num = 0.057000386 Billion write speed all = 0.568656668 MB/sec (with seq num) write speed all = 0.545068148 MB/sec (without seq num) 2024/11/23-14:17:23.829156 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262692: 928714 keys, 37389728 bytes, temperature: kWarm 2024/11/23-14:17:23.829229 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342643829193, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262692, "file_size": 37389728, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 4012774984, "table_properties": {"data_size": 31147664, "index_size": 1790080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003992, "raw_average_key_size": 28, "raw_value_size": 108213768, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928714, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342640, "oldest_key_time": 0, "file_creation_time": 1732342640, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262692, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:23.870710 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262693.sst) /dev/shm/ToplingTemp/Topling-69818-mLEjIw 2024/11/23-14:17:24.907519 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262693.sst): first pass time = 1.04's, 129.458'MB/sec 2024/11/23-14:17:26.226477 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262693.sst): old prealloc_size = 147639500, real_size = 32835440 2024/11/23-14:17:26.227094 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262693.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931710 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.06 avg-zval = 33.08 2024/11/23-14:17:26.227890 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262693.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:27.091236 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262693.sst): second pass time = 1.32's, 82.035'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.861'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2999.249'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 289.348'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33723.448'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.035'MB/sec zip pipeline throughput = 82.035'MB/sec entries = 931710 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.06 avg-zval = 33.08 usrkeys = 931710 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.06 avg-zval = 33.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931710 bytes = 3377456 } seq expand size = 6521970 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4448 value = 3.5086 dict = 1.00 all = 3.6213 } Zip/UnZip{ index = 0.0957 value = 0.2850 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 9.536874 GB avg = 0.165 KB (by entry num) total key len = 1.596891 GB avg = 0.028 KB total ukey len = 1.109357 GB avg = 0.016 KB total ukey num = 0.071124367 Billion total entry num = 0.057932096 Billion write speed all = 0.575838433 MB/sec (with seq num) write speed all = 0.551868445 MB/sec (without seq num) 2024/11/23-14:17:27.138507 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262693: 931710 keys, 37063778 bytes, temperature: kWarm 2024/11/23-14:17:27.138581 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342647138542, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262693, "file_size": 37063778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012776021, "table_properties": {"data_size": 30818272, "index_size": 1784064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26087880, "raw_average_key_size": 28, "raw_value_size": 108129872, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931710, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342643, "oldest_key_time": 0, "file_creation_time": 1732342643, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262693, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:27.182343 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262694.sst) /dev/shm/ToplingTemp/Topling-69818-m8Lhlv 2024/11/23-14:17:28.121411 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262694.sst): first pass time = 0.94's, 142.934'MB/sec 2024/11/23-14:17:29.438884 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262694.sst): old prealloc_size = 147639500, real_size = 33068976 2024/11/23-14:17:29.439498 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262694.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927513 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.71 avg-zval = 33.50 2024/11/23-14:17:29.440321 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262694.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:30.295524 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262694.sst): second pass time = 1.32's, 82.208'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.661'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2992.358'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 292.963'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34385.649'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.208'MB/sec zip pipeline throughput = 82.208'MB/sec entries = 927513 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.71 avg-zval = 33.50 usrkeys = 927513 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.71 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927513 bytes = 3478192 } seq expand size = 6492591 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5143 value = 3.4837 dict = 1.00 all = 3.5886 } Zip/UnZip{ index = 0.0951 value = 0.2871 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 9.651614 GB avg = 0.164 KB (by entry num) total key len = 1.622862 GB avg = 0.028 KB total ukey len = 1.127907 GB avg = 0.016 KB total ukey num = 0.072051880 Billion total entry num = 0.058859609 Billion write speed all = 0.583019353 MB/sec (with seq num) write speed all = 0.558669632 MB/sec (without seq num) 2024/11/23-14:17:30.341205 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262694: 927513 keys, 37401226 bytes, temperature: kWarm 2024/11/23-14:17:30.341278 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342650341241, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262694, "file_size": 37401226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012774830, "table_properties": {"data_size": 31072608, "index_size": 1764288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25970364, "raw_average_key_size": 28, "raw_value_size": 108247402, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927513, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342647, "oldest_key_time": 0, "file_creation_time": 1732342647, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262694, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:30.379258 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262695.sst) /dev/shm/ToplingTemp/Topling-69818-efxUFu 2024/11/23-14:17:31.401348 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262695.sst): first pass time = 1.02's, 131.323'MB/sec 2024/11/23-14:17:32.708384 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262695.sst): old prealloc_size = 147639500, real_size = 33635104 2024/11/23-14:17:32.708956 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262695.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933300 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0316 GB avg-val = 115.81 avg-zval = 33.87 2024/11/23-14:17:32.710450 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262695.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:33.531090 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262695.sst): second pass time = 1.31's, 82.791'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 45.520'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3253.872'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 287.620'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33908.542'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.791'MB/sec zip pipeline throughput = 82.791'MB/sec entries = 933300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.81 avg-zval = 33.87 usrkeys = 933300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.44 avg-zval = 33.87 TagRS{ kinds = RS_Key0_TagN bytes = 145920 } TagArray{ size = 750303 bytes = 2813648 } seq expand size = 5252121 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4450 value = 3.4189 dict = 1.00 all = 3.5840 } Zip/UnZip{ index = 0.0957 value = 0.2925 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 9.764951 GB avg = 0.163 KB (by entry num) total key len = 1.648994 GB avg = 0.028 KB total ukey len = 1.146573 GB avg = 0.016 KB total ukey num = 0.072985180 Billion total entry num = 0.059792909 Billion write speed all = 0.590132808 MB/sec (with seq num) write speed all = 0.565401127 MB/sec (without seq num) 2024/11/23-14:17:33.576522 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262695: 933300 keys, 37448988 bytes, temperature: kWarm 2024/11/23-14:17:33.576592 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342653576556, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262695, "file_size": 37448988, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776039, "table_properties": {"data_size": 31614544, "index_size": 1787072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26132400, "raw_average_key_size": 28, "raw_value_size": 108085460, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342650, "oldest_key_time": 0, "file_creation_time": 1732342650, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262695, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:33.616688 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262696.sst) /dev/shm/ToplingTemp/Topling-69818-IjLWMu 2024/11/23-14:17:34.612524 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262696.sst): first pass time = 1.00's, 134.784'MB/sec 2024/11/23-14:17:35.937586 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262696.sst): old prealloc_size = 147639500, real_size = 32989424 2024/11/23-14:17:35.938184 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262696.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.49 avg-zval = 33.36 2024/11/23-14:17:35.939091 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262696.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:36.790779 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262696.sst): second pass time = 1.32's, 81.739'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.614'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3083.669'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 291.472'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34621.453'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.739'MB/sec zip pipeline throughput = 81.739'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 33.36 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 33.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5141 value = 3.4917 dict = 1.00 all = 3.6070 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 9.879662 GB avg = 0.163 KB (by entry num) total key len = 1.675003 GB avg = 0.028 KB total ukey len = 1.165151 GB avg = 0.016 KB total ukey num = 0.073914085 Billion total entry num = 0.060721814 Billion write speed all = 0.597307765 MB/sec (with seq num) write speed all = 0.572196101 MB/sec (without seq num) 2024/11/23-14:17:36.836509 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262696: 928905 keys, 37210054 bytes, temperature: kWarm 2024/11/23-14:17:36.836579 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342656836544, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262696, "file_size": 37210054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 30990048, "index_size": 1766976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342653, "oldest_key_time": 0, "file_creation_time": 1732342653, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262696, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:36.875857 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262697.sst) /dev/shm/ToplingTemp/Topling-69818-UUVfdx 2024/11/23-14:17:37.908586 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262697.sst): first pass time = 1.03's, 129.969'MB/sec 2024/11/23-14:17:39.191892 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262697.sst): old prealloc_size = 147639500, real_size = 32430832 2024/11/23-14:17:39.192478 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262697.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934353 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.65 avg-zval = 32.56 2024/11/23-14:17:39.193257 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262697.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:40.057786 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262697.sst): second pass time = 1.28's, 84.278'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.349'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3151.900'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.317'MB/sec write SST data time = 0.13's, 285.604'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35192.337'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.278'MB/sec zip pipeline throughput = 84.278'MB/sec entries = 934353 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.65 avg-zval = 32.56 usrkeys = 934353 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.65 avg-zval = 32.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934353 bytes = 3387040 } seq expand size = 6540471 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5129 value = 3.5522 dict = 1.00 all = 3.6603 } Zip/UnZip{ index = 0.0951 value = 0.2815 dict = 1.00 all = 0.2732 } ---------------------------- total value len = 9.994259 GB avg = 0.162 KB (by entry num) total key len = 1.701165 GB avg = 0.028 KB total ukey len = 1.183838 GB avg = 0.016 KB total ukey num = 0.074848438 Billion total entry num = 0.061656167 Billion write speed all = 0.604482047 MB/sec (with seq num) write speed all = 0.578988283 MB/sec (without seq num) 2024/11/23-14:17:40.102906 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262697: 934353 keys, 36668482 bytes, temperature: kWarm 2024/11/23-14:17:40.102991 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342660102939, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262697, "file_size": 36668482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30419552, "index_size": 1777536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26161884, "raw_average_key_size": 28, "raw_value_size": 108055926, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934353, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342656, "oldest_key_time": 0, "file_creation_time": 1732342656, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262697, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:40.150343 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262698.sst) /dev/shm/ToplingTemp/Topling-69818-s5VVhw 2024/11/23-14:17:41.101290 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262698.sst): first pass time = 0.95's, 141.149'MB/sec 2024/11/23-14:17:42.432148 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262698.sst): old prealloc_size = 147639500, real_size = 34336736 2024/11/23-14:17:42.432713 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262698.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930550 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.23 avg-zval = 34.73 2024/11/23-14:17:42.434258 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262698.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:43.283437 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262698.sst): second pass time = 1.33's, 81.351'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.299'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3290.059'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 290.387'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29019.271'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.351'MB/sec zip pipeline throughput = 81.351'MB/sec entries = 930550 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.23 avg-zval = 34.73 usrkeys = 930550 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.23 avg-zval = 34.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930550 bytes = 3373264 } seq expand size = 6513850 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4434 value = 3.3464 dict = 1.00 all = 3.4806 } Zip/UnZip{ index = 0.0958 value = 0.2988 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 10.108935 GB avg = 0.162 KB (by entry num) total key len = 1.727221 GB avg = 0.028 KB total ukey len = 1.202449 GB avg = 0.016 KB total ukey num = 0.075778988 Billion total entry num = 0.062586717 Billion write speed all = 0.611653839 MB/sec (with seq num) write speed all = 0.585779623 MB/sec (without seq num) 2024/11/23-14:17:43.330113 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262698: 930550 keys, 38561398 bytes, temperature: kWarm 2024/11/23-14:17:43.330180 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342663330144, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262698, "file_size": 38561398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 4012776268, "table_properties": {"data_size": 32321808, "index_size": 1782080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26055400, "raw_average_key_size": 28, "raw_value_size": 108162448, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930550, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342660, "oldest_key_time": 0, "file_creation_time": 1732342660, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262698, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:43.368600 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262699.sst) /dev/shm/ToplingTemp/Topling-69818-NrvRYx 2024/11/23-14:17:44.186696 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262699.sst): first pass time = 0.82's, 164.068'MB/sec 2024/11/23-14:17:45.479853 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262699.sst): old prealloc_size = 147639500, real_size = 31411024 2024/11/23-14:17:45.480454 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262699.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931281 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0294 GB avg-val = 116.12 avg-zval = 31.56 2024/11/23-14:17:45.482044 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262699.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:46.318628 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262699.sst): second pass time = 1.29's, 83.708'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.628'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3080.394'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 275.284'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32816.967'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.708'MB/sec zip pipeline throughput = 83.708'MB/sec entries = 931281 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.12 avg-zval = 31.56 usrkeys = 931281 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.12 avg-zval = 31.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931281 bytes = 3492320 } seq expand size = 6518967 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.4460 value = 3.6789 dict = 1.00 all = 3.7538 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2664 } ---------------------------- total value len = 10.223596 GB avg = 0.161 KB (by entry num) total key len = 1.753296 GB avg = 0.028 KB total ukey len = 1.221075 GB avg = 0.016 KB total ukey num = 0.076710269 Billion total entry num = 0.063517998 Billion write speed all = 0.618829592 MB/sec (with seq num) write speed all = 0.592574489 MB/sec (without seq num) 2024/11/23-14:17:46.363853 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262699: 931281 keys, 35754832 bytes, temperature: kWarm 2024/11/23-14:17:46.363923 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342666363888, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262699, "file_size": 35754832, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776283, "table_properties": {"data_size": 29395008, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26075868, "raw_average_key_size": 28, "raw_value_size": 108141910, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931281, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342663, "oldest_key_time": 0, "file_creation_time": 1732342663, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262699, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:46.400083 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262700.sst) /dev/shm/ToplingTemp/Topling-69818-xrCuUv 2024/11/23-14:17:47.449015 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262700.sst): first pass time = 1.05's, 127.961'MB/sec 2024/11/23-14:17:48.737142 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262700.sst): old prealloc_size = 147639500, real_size = 30305504 2024/11/23-14:17:48.737714 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262700.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930678 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0283 GB avg-val = 116.22 avg-zval = 30.39 2024/11/23-14:17:48.738540 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262700.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:49.562592 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262700.sst): second pass time = 1.29's, 84.015'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.972'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3245.097'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 269.338'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36928.767'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.015'MB/sec zip pipeline throughput = 84.015'MB/sec entries = 930678 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.22 avg-zval = 30.39 usrkeys = 930678 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.22 avg-zval = 30.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930678 bytes = 3490064 } seq expand size = 6514746 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.3767 value = 3.8247 dict = 1.00 all = 3.8741 } Zip/UnZip{ index = 0.0964 value = 0.2615 dict = 1.00 all = 0.2581 } ---------------------------- total value len = 10.338269 GB avg = 0.160 KB (by entry num) total key len = 1.779355 GB avg = 0.028 KB total ukey len = 1.239688 GB avg = 0.016 KB total ukey num = 0.077640947 Billion total entry num = 0.064448676 Billion write speed all = 0.625996122 MB/sec (with seq num) write speed all = 0.599360789 MB/sec (without seq num) 2024/11/23-14:17:49.606368 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262700: 930678 keys, 34645184 bytes, temperature: kWarm 2024/11/23-14:17:49.606446 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342669606410, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262700, "file_size": 34645184, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3428974320, "largest_seqno": 4012776300, "table_properties": {"data_size": 28278864, "index_size": 1793792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058984, "raw_average_key_size": 28, "raw_value_size": 108158752, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930678, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342666, "oldest_key_time": 0, "file_creation_time": 1732342666, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262700, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:49.644520 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262701.sst) /dev/shm/ToplingTemp/Topling-69818-212uVw 2024/11/23-14:17:50.554920 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262701.sst): first pass time = 0.91's, 147.434'MB/sec 2024/11/23-14:17:51.902371 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262701.sst): old prealloc_size = 147639500, real_size = 34242192 2024/11/23-14:17:51.903048 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262701.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922415 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0322 GB avg-val = 117.51 avg-zval = 34.96 2024/11/23-14:17:51.903928 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262701.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:52.774077 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262701.sst): second pass time = 1.35's, 80.487'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 44.103'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2749.884'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.14's, 283.493'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35831.123'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.487'MB/sec zip pipeline throughput = 80.487'MB/sec entries = 922415 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.51 avg-zval = 34.96 usrkeys = 922415 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.51 avg-zval = 34.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922415 bytes = 3343776 } seq expand size = 6456905 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4459 value = 3.3614 dict = 1.00 all = 3.4912 } Zip/UnZip{ index = 0.0957 value = 0.2975 dict = 1.00 all = 0.2864 } ---------------------------- total value len = 10.453116 GB avg = 0.160 KB (by entry num) total key len = 1.805183 GB avg = 0.028 KB total ukey len = 1.258137 GB avg = 0.016 KB total ukey num = 0.078563362 Billion total entry num = 0.065371091 Billion write speed all = 0.633158327 MB/sec (with seq num) write speed all = 0.606146260 MB/sec (without seq num) 2024/11/23-14:17:52.822741 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262701: 922415 keys, 38444522 bytes, temperature: kWarm 2024/11/23-14:17:52.822811 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342672822775, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262701, "file_size": 38444522, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 32245312, "index_size": 1766080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25827620, "raw_average_key_size": 28, "raw_value_size": 108390158, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922415, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342669, "oldest_key_time": 0, "file_creation_time": 1732342669, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262701, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:52.862536 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262702.sst) /dev/shm/ToplingTemp/Topling-69818-pgWJ7w 2024/11/23-14:17:53.711249 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262702.sst): first pass time = 0.85's, 158.149'MB/sec 2024/11/23-14:17:55.351280 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262702.sst): old prealloc_size = 147639500, real_size = 32083584 2024/11/23-14:17:55.351821 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262702.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931530 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0301 GB avg-val = 116.08 avg-zval = 32.29 2024/11/23-14:17:55.352654 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262702.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:56.350816 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262702.sst): second pass time = 1.64's, 65.964'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 36.446'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3403.855'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.269'MB/sec write SST data time = 0.13's, 270.709'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38854.559'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.964'MB/sec zip pipeline throughput = 65.964'MB/sec entries = 931530 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.08 avg-zval = 32.29 usrkeys = 931530 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.08 avg-zval = 32.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931530 bytes = 3493248 } seq expand size = 6520710 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5126 value = 3.5951 dict = 1.00 all = 3.6845 } Zip/UnZip{ index = 0.0951 value = 0.2782 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 10.567772 GB avg = 0.159 KB (by entry num) total key len = 1.831266 GB avg = 0.028 KB total ukey len = 1.276767 GB avg = 0.016 KB total ukey num = 0.079494892 Billion total entry num = 0.066302621 Billion write speed all = 0.640309375 MB/sec (with seq num) write speed all = 0.612917450 MB/sec (without seq num) 2024/11/23-14:17:56.395603 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262702: 931530 keys, 36428054 bytes, temperature: kWarm 2024/11/23-14:17:56.395672 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342676395638, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262702, "file_size": 36428054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776331, "table_properties": {"data_size": 30078320, "index_size": 1772224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26082840, "raw_average_key_size": 28, "raw_value_size": 108134956, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931530, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342672, "oldest_key_time": 0, "file_creation_time": 1732342672, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262702, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:56.434587 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262703.sst) /dev/shm/ToplingTemp/Topling-69818-oogstw 2024/11/23-14:17:57.407289 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262703.sst): first pass time = 0.97's, 137.990'MB/sec 2024/11/23-14:17:58.782665 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262703.sst): old prealloc_size = 147639500, real_size = 32033360 2024/11/23-14:17:58.783232 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262703.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 116.15 avg-zval = 32.24 2024/11/23-14:17:58.784189 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262703.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:17:59.829644 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262703.sst): second pass time = 1.37's, 78.669'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 34.746'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3243.718'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.24's, 154.454'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35413.375'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.669'MB/sec zip pipeline throughput = 78.669'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 32.24 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 32.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4434 value = 3.6028 dict = 1.00 all = 3.6897 } Zip/UnZip{ index = 0.0958 value = 0.2776 dict = 1.00 all = 0.2710 } ---------------------------- total value len = 10.682437 GB avg = 0.159 KB (by entry num) total key len = 1.857336 GB avg = 0.028 KB total ukey len = 1.295389 GB avg = 0.016 KB total ukey num = 0.080425975 Billion total entry num = 0.067233704 Billion write speed all = 0.647460897 MB/sec (with seq num) write speed all = 0.619689298 MB/sec (without seq num) 2024/11/23-14:17:59.878638 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262703: 931083 keys, 36376784 bytes, temperature: kWarm 2024/11/23-14:17:59.878713 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342679878667, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262703, "file_size": 36376784, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30017280, "index_size": 1783104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342676, "oldest_key_time": 0, "file_creation_time": 1732342676, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262703, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:17:59.951560 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262704.sst) /dev/shm/ToplingTemp/Topling-69818-rmrq0u 2024/11/23-14:18:00.884661 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262704.sst): first pass time = 0.93's, 143.846'MB/sec 2024/11/23-14:18:03.897533 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262704.sst): old prealloc_size = 147639500, real_size = 31964000 2024/11/23-14:18:03.898112 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262704.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925740 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0299 GB avg-val = 116.98 avg-zval = 32.35 2024/11/23-14:18:03.898882 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262704.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:18:04.965892 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262704.sst): second pass time = 3.01's, 35.953'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 33.657'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3221.241'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.277'MB/sec write SST data time = 0.23's, 153.960'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37092.076'MB/sec zip my value time = 3.01's, unzip length = 0.108'GB zip my value throughput = 35.953'MB/sec zip pipeline throughput = 93.541'MB/sec entries = 925740 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.98 avg-zval = 32.35 usrkeys = 925740 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.22 avg-zval = 32.35 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 824262 bytes = 2987968 } seq expand size = 5769834 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.3760 value = 3.6162 dict = 1.00 all = 3.7328 } Zip/UnZip{ index = 0.0964 value = 0.2765 dict = 1.00 all = 0.2679 } ---------------------------- total value len = 10.796504 GB avg = 0.158 KB (by entry num) total key len = 1.883257 GB avg = 0.028 KB total ukey len = 1.313904 GB avg = 0.016 KB total ukey num = 0.081351715 Billion total entry num = 0.068159444 Billion write speed all = 0.654515248 MB/sec (with seq num) write speed all = 0.626368727 MB/sec (without seq num) 2024/11/23-14:18:05.019290 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262704: 925740 keys, 35956274 bytes, temperature: kWarm 2024/11/23-14:18:05.019379 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342685019333, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262704, "file_size": 35956274, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 29947984, "index_size": 1784384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920720, "raw_average_key_size": 28, "raw_value_size": 108297148, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925740, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342679, "oldest_key_time": 0, "file_creation_time": 1732342679, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262704, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:18:05.065343 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262705.sst) /dev/shm/ToplingTemp/Topling-69818-xUTnzv 2024/11/23-14:18:05.704073 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262705.sst): first pass time = 0.64's, 210.142'MB/sec 2024/11/23-14:18:08.868538 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262705.sst): old prealloc_size = 147639500, real_size = 36903184 2024/11/23-14:18:08.869134 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262705.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926242 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.91 avg-zval = 37.68 2024/11/23-14:18:08.869349 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262705.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:18:09.800367 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262705.sst): second pass time = 3.16's, 34.227'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 41.019'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3086.940'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.23's, 163.492'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 38751.112'MB/sec zip my value time = 3.16's, unzip length = 0.108'GB zip my value throughput = 34.227'MB/sec zip pipeline throughput = 102.106'MB/sec entries = 926242 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.91 avg-zval = 37.68 usrkeys = 926242 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.67 avg-zval = 37.68 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 100732 bytes = 314800 } seq expand size = 705124 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4446 value = 3.1029 dict = 1.00 all = 3.5116 } Zip/UnZip{ index = 0.0957 value = 0.3223 dict = 1.00 all = 0.2848 } ---------------------------- total value len = 10.905492 GB avg = 0.158 KB (by entry num) total key len = 1.909192 GB avg = 0.028 KB total ukey len = 1.332428 GB avg = 0.016 KB total ukey num = 0.082277957 Billion total entry num = 0.069085686 Billion write speed all = 0.661314787 MB/sec (with seq num) write speed all = 0.632792891 MB/sec (without seq num) 2024/11/23-14:18:09.849869 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262705: 926242 keys, 38221041 bytes, temperature: kWarm 2024/11/23-14:18:09.850190 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342689850140, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262705, "file_size": 38221041, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3269812299, "table_properties": {"data_size": 34897792, "index_size": 1773632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934776, "raw_average_key_size": 28, "raw_value_size": 108283004, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342685, "oldest_key_time": 0, "file_creation_time": 1732342685, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262705, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:18:09.932281 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262706.sst) /dev/shm/ToplingTemp/Topling-69818-O3nB0u 2024/11/23-14:18:10.460671 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262706.sst): first pass time = 0.53's, 161.009'MB/sec 2024/11/23-14:18:12.881655 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262706.sst): old prealloc_size = 147639500, real_size = 24322736 2024/11/23-14:18:12.882038 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262706.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 588957 prefix = 4 raw-key = 0.0094 GB zip-key = 0.0011 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0686 GB zip-val = 0.0230 GB avg-val = 116.44 avg-zval = 39.13 2024/11/23-14:18:12.882320 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262706.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0015 GB, waited 0.000 sec, Key+Value = 0.085 GB 2024/11/23-14:18:13.341532 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00/262706.sst): second pass time = 2.42's, 28.334'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.46's, 56.032'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3113.316'MB/sec (index lex order gen) rebuild zvType time = 0.38's, 0.390'MB/sec write SST data time = 0.08's, 313.722'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.689'MB, throughput = 22785.114'MB/sec zip my value time = 2.42's, unzip length = 0.069'GB zip my value throughput = 28.334'MB/sec zip pipeline throughput = 118.501'MB/sec entries = 588957 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.44 avg-zval = 39.13 usrkeys = 588957 avg-key = 20.00 avg-zkey = 1.92 avg-val = 118.98 avg-zval = 39.13 TagRS{ kinds = RS_Key0_TagN bytes = 92120 } TagArray{ size = 214352 bytes = 803840 } seq expand size = 1500464 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0118 GB value = 0.0686 GB dict = 0.69 MB all = 0.0851 GB } __ZipSize{ index = 0.0011 GB value = 0.0230 GB dict = 0.69 MB all = 0.0258 GB } UnZip/Zip{ index = 10.4389 value = 2.9755 dict = 1.00 all = 3.3014 } Zip/UnZip{ index = 0.0958 value = 0.3361 dict = 1.00 all = 0.3029 } ---------------------------- total value len = 10.975569 GB avg = 0.158 KB (by entry num) total key len = 1.925682 GB avg = 0.028 KB total ukey len = 1.344208 GB avg = 0.016 KB total ukey num = 0.082866914 Billion total entry num = 0.069674643 Billion write speed all = 0.665660543 MB/sec (with seq num) write speed all = 0.636900753 MB/sec (without seq num) 2024/11/23-14:18:13.376812 139957542479424 [compaction_job.cc:1948] [default] [JOB 27488] Generated table #262706: 588957 keys, 25766761 bytes, temperature: kWarm 2024/11/23-14:18:13.376880 139957542479424 EVENT_LOG_v1 {"time_micros": 1732342693376845, "cf_name": "default", "job": 27488, "event": "table_file_creation", "file_number": 262706, "file_size": 25766761, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4013799017, "table_properties": {"data_size": 23046944, "index_size": 1128384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 16490796, "raw_average_key_size": 28, "raw_value_size": 68576230, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 588957, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342689, "oldest_key_time": 0, "file_creation_time": 1732342689, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262706, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:18:13.399085 139957542479424 [compaction_job.cc:703] job-27488: subcompact[0], size: 0.692728 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27488/att-00 [262688,262689,262690,262691,262692,262693,262694,262695,262696,262697,262698,262699,262700,262701,262702,262703,262704,262705,262706] 2024/11/23-14:18:13.520502 139957542479424 (Original Log Time 2024/11/23-14:17:07.122407) EVENT_LOG_v1 {"time_micros": 1732342627122379, "job": 27488, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [262619], "files_L3": [262684, 262685, 262588, 262589, 262590, 262591, 262566, 262567, 262568, 262569, 262570, 262571, 262572, 262573, 262574, 262575, 262576, 262577, 262578, 206445, 206449, 204580, 204186, 203758, 203310], "score": 0.997004, "input_data_size": 704652306, "oldest_snapshot_seqno": 4014770534} 2024/11/23-14:18:13.520506 139957542479424 (Original Log Time 2024/11/23-14:18:13.518855) [compaction_job.cc:2022] [default] [JOB 27488] Compacted 1@2 + 25@3 files to L3 => 692728332 bytes 2024/11/23-14:18:13.520508 139957542479424 (Original Log Time 2024/11/23-14:18:13.520424) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 19 0 0 0] max score 0.03, MB/sec: 10.6 rd, 10.5 wr, level 3, files in(1, 25) out(19 +0 blob) MB in(27.1, 644.9 +0.0 blob) out(660.6 +0.0 blob), read-write-amplify(49.1) write-amplify(24.4) OK, records in: 17334653, records dropped: 18174 output_compression: Snappy 2024/11/23-14:18:13.520511 139957542479424 (Original Log Time 2024/11/23-14:18:13.520474) EVENT_LOG_v1 {"time_micros": 1732342693520449, "cf": "default", "job": 27488, "event": "compaction_finished", "compaction_time_micros": 66276688, "compaction_time_cpu_micros": 26678926, "output_level": 3, "num_output_files": 19, "total_output_size": 692728332, "num_input_records": 17334653, "num_output_records": 17316479, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 507165923, "file_range_sync_nanos": 0, "file_fsync_nanos": 14942, "file_prepare_write_nanos": 51062, "lsm_state": [0, 0, 0, 19, 0, 0, 0]} 2024/11/23-14:18:13.649298 139957542479424 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732342626,"executesMs":66547,"compactionJobId":27488,"attempt":0,"compactionInputRawBytes":2503568070,"compactionInputZipBytes":704652306,"compactionOutputRawBytes":0,"compactionOutputZipBytes":692728332,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-3b804a241d9784369a57fbeb11a4cea3-88cb34d51585d620-00"}}, response = 2024/11/23-14:18:13.649393 139957542479424 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27488/att-00: olev 3, work 66.547 s, result 7.794 ms, install 1.673 ms, input{raw 2.332 GiB zip 672.009 MiB}