2024/11/22-23:40:08.289968 140024902895168 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00: fork to child time = 0.009067 sec 2024/11/22-23:40:08.290769 140024902895168 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00 2024/11/22-23:40:08.297544 140024902895168 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00 2024/11/22-23:40:08.298658 140024902895168 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-23:40:08.298933 140024902895168 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-23:40:08.349065 140024902895168 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-14231/att-00/MANIFEST-206347 2024/11/22-23:40:08.380944 140024902895168 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-14231/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 234340, last_sequence is 3649420645, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 234196 2024/11/22-23:40:08.380953 140024902895168 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-23:40:08.381045 140024902895168 [version_set.cc:5721] Creating manifest 234340 2024/11/22-23:40:08.419511 140024902895168 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-23:40:08.439598 140024902895168 [compaction_job.cc:2437] [default] [JOB 14231] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-23:40:08.439616 140024902895168 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 10670 Base level 2, inputs: [234337(27MB)], [234289(34MB) 234290(35MB) 234291(37MB) 234292(35MB) 234293(36MB) 234294(34MB) 234295(36MB) 234303(36MB) 234304(35MB) 234305(35MB) 234306(35MB) 234307(35MB) 234308(33MB) 234309(36MB) 234310(35MB)] 2024/11/22-23:40:08.446193 140024902895168 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:40:08.446225 140024902895168 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:40:08.446246 140024902895168 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:40:08.446251 140024902895168 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:40:08.446262 140024902895168 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:40:08.918118 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234341.sst) /dev/shm/ToplingTemp/Topling-336247-tZsSxA 2024/11/22-23:40:09.794947 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234341.sst): first pass time = 0.88's, 153.080'MB/sec 2024/11/22-23:40:11.390922 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234341.sst): old prealloc_size = 147639500, real_size = 32474736 2024/11/22-23:40:11.391556 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234341.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925305 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 117.05 avg-zval = 32.93 2024/11/22-23:40:11.392425 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234341.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:12.974181 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234341.sst): second pass time = 1.59's, 67.932'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.58's, 23.097'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2890.010'MB/sec (index lex order gen) rebuild zvType time = 1.25's, 0.185'MB/sec write SST data time = 0.33's, 109.672'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35484.896'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.932'MB/sec zip pipeline throughput = 67.932'MB/sec entries = 925305 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.05 avg-zval = 32.93 usrkeys = 925305 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.05 avg-zval = 32.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925305 bytes = 3238576 } seq expand size = 6477135 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4453 value = 3.5544 dict = 1.00 all = 3.6703 } Zip/UnZip{ index = 0.0957 value = 0.2813 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 0.114786 GB avg = 0.124 KB (by entry num) total key len = 0.025909 GB avg = 0.028 KB total ukey len = 0.018506 GB avg = 0.020 KB total ukey num = 0.000925305 Billion total entry num = 0.000925305 Billion write speed all = 0.002641173 MB/sec (with seq num) write speed all = 0.002502212 MB/sec (without seq num) 2024/11/22-23:40:13.015982 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234341: 925305 keys, 36568552 bytes, temperature: kWarm 2024/11/22-23:40:13.016097 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290013016026, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234341, "file_size": 36568552, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3499102480, "largest_seqno": 3649396711, "table_properties": {"data_size": 30471520, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25908540, "raw_average_key_size": 28, "raw_value_size": 108309302, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925305, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290008, "oldest_key_time": 0, "file_creation_time": 1732290008, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234341, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:13.056007 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234342.sst) /dev/shm/ToplingTemp/Topling-336247-2LGreE 2024/11/22-23:40:13.922885 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234342.sst): first pass time = 0.87's, 154.837'MB/sec 2024/11/22-23:40:15.317707 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234342.sst): old prealloc_size = 147639500, real_size = 32478672 2024/11/22-23:40:15.318282 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234342.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925406 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 117.04 avg-zval = 32.93 2024/11/22-23:40:15.319083 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234342.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:16.798865 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234342.sst): second pass time = 1.39's, 77.684'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.48's, 24.773'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3186.147'MB/sec (index lex order gen) rebuild zvType time = 1.17's, 0.198'MB/sec write SST data time = 0.31's, 116.983'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36054.516'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.684'MB/sec zip pipeline throughput = 77.684'MB/sec entries = 925406 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.04 avg-zval = 32.93 usrkeys = 925406 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.04 avg-zval = 32.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925406 bytes = 3354608 } seq expand size = 6477842 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4431 value = 3.5540 dict = 1.00 all = 3.6580 } Zip/UnZip{ index = 0.0958 value = 0.2814 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 0.229571 GB avg = 0.124 KB (by entry num) total key len = 0.051820 GB avg = 0.028 KB total ukey len = 0.037014 GB avg = 0.020 KB total ukey num = 0.001850711 Billion total entry num = 0.001850711 Billion write speed all = 0.005281980 MB/sec (with seq num) write speed all = 0.005004063 MB/sec (without seq num) 2024/11/22-23:40:16.856373 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234342: 925406 keys, 36691814 bytes, temperature: kWarm 2024/11/22-23:40:16.856436 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290016856405, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234342, "file_size": 36691814, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3649396722, "table_properties": {"data_size": 30474880, "index_size": 1772288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911368, "raw_average_key_size": 28, "raw_value_size": 108306468, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925406, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290013, "oldest_key_time": 0, "file_creation_time": 1732290013, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234342, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:16.897029 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234343.sst) /dev/shm/ToplingTemp/Topling-336247-aq6bXC 2024/11/22-23:40:18.300506 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234343.sst): first pass time = 1.40's, 95.635'MB/sec 2024/11/22-23:40:20.677156 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234343.sst): old prealloc_size = 147639500, real_size = 37220480 2024/11/22-23:40:20.677724 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234343.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926044 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0352 GB avg-val = 116.94 avg-zval = 38.04 2024/11/22-23:40:20.678069 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234343.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:21.662389 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234343.sst): second pass time = 2.38's, 45.577'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 39.781'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3216.271'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.20's, 197.884'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33313.870'MB/sec zip my value time = 2.38's, unzip length = 0.108'GB zip my value throughput = 45.577'MB/sec zip pipeline throughput = 141.416'MB/sec entries = 926044 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 38.04 usrkeys = 926044 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.19 avg-zval = 38.04 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 298091 bytes = 968816 } seq expand size = 2086637 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5133 value = 3.0740 dict = 1.00 all = 3.4246 } Zip/UnZip{ index = 0.0951 value = 0.3253 dict = 1.00 all = 0.2920 } ---------------------------- total value len = 0.339946 GB avg = 0.122 KB (by entry num) total key len = 0.077749 GB avg = 0.028 KB total ukey len = 0.055535 GB avg = 0.020 KB total ukey num = 0.002776755 Billion total entry num = 0.002776755 Billion write speed all = 0.007839833 MB/sec (with seq num) write speed all = 0.007422892 MB/sec (without seq num) 2024/11/22-23:40:21.701974 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234343: 926044 keys, 39192745 bytes, temperature: kWarm 2024/11/22-23:40:21.702053 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290021702009, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234343, "file_size": 39192745, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 35227120, "index_size": 1761664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25929232, "raw_average_key_size": 28, "raw_value_size": 108288540, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926044, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290016, "oldest_key_time": 0, "file_creation_time": 1732290016, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234343, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:21.760085 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234344.sst) /dev/shm/ToplingTemp/Topling-336247-qAitSE 2024/11/22-23:40:22.642964 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234344.sst): first pass time = 0.88's, 152.031'MB/sec 2024/11/22-23:40:24.740020 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234344.sst): old prealloc_size = 147639500, real_size = 36470128 2024/11/22-23:40:24.740639 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234344.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927086 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.20 2024/11/22-23:40:24.740693 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234344.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:25.389936 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234344.sst): second pass time = 2.10's, 51.660'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 57.437'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2912.747'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.450'MB/sec write SST data time = 0.13's, 277.869'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37490.112'MB/sec zip my value time = 2.10's, unzip length = 0.108'GB zip my value throughput = 51.660'MB/sec zip pipeline throughput = 51.660'MB/sec entries = 927086 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.20 usrkeys = 927086 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5816 value = 3.1392 dict = 1.00 all = 3.5956 } Zip/UnZip{ index = 0.0945 value = 0.3185 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 0.448205 GB avg = 0.121 KB (by entry num) total key len = 0.103708 GB avg = 0.028 KB total ukey len = 0.074077 GB avg = 0.020 KB total ukey num = 0.003703841 Billion total entry num = 0.003703841 Billion write speed all = 0.010358278 MB/sec (with seq num) write speed all = 0.009802169 MB/sec (without seq num) 2024/11/22-23:40:25.421500 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234344: 927086 keys, 37328414 bytes, temperature: kWarm 2024/11/22-23:40:25.421559 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290025421528, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234344, "file_size": 37328414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34485920, "index_size": 1752256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958408, "raw_average_key_size": 28, "raw_value_size": 108259320, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927086, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290021, "oldest_key_time": 0, "file_creation_time": 1732290021, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234344, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:25.460596 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234345.sst) /dev/shm/ToplingTemp/Topling-336247-b7AtLD 2024/11/22-23:40:26.423515 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234345.sst): first pass time = 0.96's, 139.393'MB/sec 2024/11/22-23:40:28.271542 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234345.sst): old prealloc_size = 147639500, real_size = 36500944 2024/11/22-23:40:28.272116 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234345.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924385 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 117.20 avg-zval = 37.33 2024/11/22-23:40:28.272481 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234345.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:29.090883 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234345.sst): second pass time = 1.85's, 58.664'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 46.974'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3196.080'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.384'MB/sec write SST data time = 0.22's, 177.271'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 27256.747'MB/sec zip my value time = 1.85's, unzip length = 0.108'GB zip my value throughput = 58.664'MB/sec zip pipeline throughput = 112.675'MB/sec entries = 924385 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.20 avg-zval = 37.33 usrkeys = 924385 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.48 avg-zval = 37.33 TagRS{ kinds = RS_Key0_TagN bytes = 144520 } TagArray{ size = 301653 bytes = 980384 } seq expand size = 2111571 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5120 value = 3.1392 dict = 1.00 all = 3.4875 } Zip/UnZip{ index = 0.0951 value = 0.3186 dict = 1.00 all = 0.2867 } ---------------------------- total value len = 0.558652 GB avg = 0.121 KB (by entry num) total key len = 0.129590 GB avg = 0.028 KB total ukey len = 0.092565 GB avg = 0.020 KB total ukey num = 0.004628226 Billion total entry num = 0.004628226 Billion write speed all = 0.012916006 MB/sec (with seq num) write speed all = 0.012221155 MB/sec (without seq num) 2024/11/22-23:40:29.127530 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234345: 924385 keys, 38485923 bytes, temperature: kWarm 2024/11/22-23:40:29.127610 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290029127567, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234345, "file_size": 38485923, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3335515830, "table_properties": {"data_size": 34510912, "index_size": 1758720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25882780, "raw_average_key_size": 28, "raw_value_size": 108335090, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924385, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290025, "oldest_key_time": 0, "file_creation_time": 1732290025, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234345, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:29.203242 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234346.sst) /dev/shm/ToplingTemp/Topling-336247-8Tv3xA 2024/11/22-23:40:30.219303 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234346.sst): first pass time = 1.02's, 132.103'MB/sec 2024/11/22-23:40:31.777117 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234346.sst): old prealloc_size = 147639500, real_size = 32070928 2024/11/22-23:40:31.777664 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234346.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925592 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 117.01 avg-zval = 32.50 2024/11/22-23:40:31.778464 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234346.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:32.858292 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234346.sst): second pass time = 1.56's, 69.550'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 33.558'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3351.490'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.272'MB/sec write SST data time = 0.23's, 158.054'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27277.413'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.550'MB/sec zip pipeline throughput = 87.778'MB/sec entries = 925592 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.01 avg-zval = 32.50 usrkeys = 925592 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.01 avg-zval = 32.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925592 bytes = 3355280 } seq expand size = 6479144 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5124 value = 3.6006 dict = 1.00 all = 3.6994 } Zip/UnZip{ index = 0.0951 value = 0.2777 dict = 1.00 all = 0.2703 } ---------------------------- total value len = 0.673432 GB avg = 0.121 KB (by entry num) total key len = 0.155507 GB avg = 0.028 KB total ukey len = 0.111076 GB avg = 0.020 KB total ukey num = 0.005553818 Billion total entry num = 0.005553818 Billion write speed all = 0.015555319 MB/sec (with seq num) write speed all = 0.014721565 MB/sec (without seq num) 2024/11/22-23:40:32.893253 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234346: 925592 keys, 36281278 bytes, temperature: kWarm 2024/11/22-23:40:32.893316 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290032893283, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234346, "file_size": 36281278, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3335515831, "largest_seqno": 3649396175, "table_properties": {"data_size": 30078400, "index_size": 1760960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25916576, "raw_average_key_size": 28, "raw_value_size": 108301292, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925592, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290029, "oldest_key_time": 0, "file_creation_time": 1732290029, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234346, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:32.930469 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234347.sst) /dev/shm/ToplingTemp/Topling-336247-ib8eiE 2024/11/22-23:40:33.799972 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234347.sst): first pass time = 0.87's, 154.368'MB/sec 2024/11/22-23:40:35.317598 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234347.sst): old prealloc_size = 147639500, real_size = 34487888 2024/11/22-23:40:35.318140 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234347.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929815 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.35 avg-zval = 34.94 2024/11/22-23:40:35.318929 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234347.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:36.372729 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234347.sst): second pass time = 1.52's, 71.329'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 36.689'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3379.657'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.251'MB/sec write SST data time = 0.13's, 302.592'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34180.868'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.329'MB/sec zip pipeline throughput = 71.329'MB/sec entries = 929815 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.35 avg-zval = 34.94 usrkeys = 929815 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.35 avg-zval = 34.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929815 bytes = 3370592 } seq expand size = 6508705 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5118 value = 3.3301 dict = 1.00 all = 3.4671 } Zip/UnZip{ index = 0.0951 value = 0.3003 dict = 1.00 all = 0.2884 } ---------------------------- total value len = 0.788124 GB avg = 0.122 KB (by entry num) total key len = 0.181542 GB avg = 0.028 KB total ukey len = 0.129673 GB avg = 0.020 KB total ukey num = 0.006483633 Billion total entry num = 0.006483633 Billion write speed all = 0.018194897 MB/sec (with seq num) write speed all = 0.017221621 MB/sec (without seq num) 2024/11/22-23:40:36.408286 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234347: 929815 keys, 38711290 bytes, temperature: kWarm 2024/11/22-23:40:36.408355 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290036408324, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234347, "file_size": 38711290, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3649396222, "table_properties": {"data_size": 32486144, "index_size": 1769088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26034820, "raw_average_key_size": 28, "raw_value_size": 108182918, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929815, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290032, "oldest_key_time": 0, "file_creation_time": 1732290032, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234347, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:36.442458 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234348.sst) /dev/shm/ToplingTemp/Topling-336247-Qe1KQC 2024/11/22-23:40:37.442441 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234348.sst): first pass time = 1.00's, 134.225'MB/sec 2024/11/22-23:40:38.857758 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234348.sst): old prealloc_size = 147639500, real_size = 33529616 2024/11/22-23:40:38.858349 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234348.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932099 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0315 GB avg-val = 116.00 avg-zval = 33.81 2024/11/22-23:40:38.859056 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234348.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:39.782243 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234348.sst): second pass time = 1.41's, 76.471'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.466'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3171.161'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.13's, 297.264'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36198.408'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.471'MB/sec zip pipeline throughput = 76.471'MB/sec entries = 932099 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.00 avg-zval = 33.81 usrkeys = 932099 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.00 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932099 bytes = 3029344 } seq expand size = 6524693 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4454 value = 3.4311 dict = 1.00 all = 3.5878 } Zip/UnZip{ index = 0.0957 value = 0.2915 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 0.902768 GB avg = 0.122 KB (by entry num) total key len = 0.207640 GB avg = 0.028 KB total ukey len = 0.148315 GB avg = 0.020 KB total ukey num = 0.007415732 Billion total entry num = 0.007415732 Billion write speed all = 0.020834470 MB/sec (with seq num) write speed all = 0.019721346 MB/sec (without seq num) 2024/11/22-23:40:39.816079 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234348: 932099 keys, 37409608 bytes, temperature: kWarm 2024/11/22-23:40:39.816137 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290039816106, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234348, "file_size": 37409608, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3649396254, "table_properties": {"data_size": 31511680, "index_size": 1784704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26098772, "raw_average_key_size": 28, "raw_value_size": 108119078, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932099, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290036, "oldest_key_time": 0, "file_creation_time": 1732290036, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234348, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:39.855710 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234349.sst) /dev/shm/ToplingTemp/Topling-336247-3csqzA 2024/11/22-23:40:40.690598 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234349.sst): first pass time = 0.83's, 160.768'MB/sec 2024/11/22-23:40:42.257509 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234349.sst): old prealloc_size = 147639500, real_size = 32806032 2024/11/22-23:40:42.258137 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234349.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926397 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 116.88 avg-zval = 33.26 2024/11/22-23:40:42.258981 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234349.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:43.557975 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234349.sst): second pass time = 1.57's, 69.136'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.30's, 28.466'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2913.506'MB/sec (index lex order gen) rebuild zvType time = 1.14's, 0.203'MB/sec write SST data time = 0.16's, 231.664'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36256.335'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 69.136'MB/sec zip pipeline throughput = 69.136'MB/sec entries = 926397 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.88 avg-zval = 33.26 usrkeys = 926397 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.88 avg-zval = 33.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926397 bytes = 3358208 } seq expand size = 6484779 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5127 value = 3.5142 dict = 1.00 all = 3.6257 } Zip/UnZip{ index = 0.0951 value = 0.2846 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 1.017531 GB avg = 0.122 KB (by entry num) total key len = 0.233580 GB avg = 0.028 KB total ukey len = 0.166843 GB avg = 0.020 KB total ukey num = 0.008342129 Billion total entry num = 0.008342129 Billion write speed all = 0.023472794 MB/sec (with seq num) write speed all = 0.022220703 MB/sec (without seq num) 2024/11/22-23:40:43.592812 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234349: 926397 keys, 37018842 bytes, temperature: kWarm 2024/11/22-23:40:43.592874 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290043592842, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234349, "file_size": 37018842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3649396586, "table_properties": {"data_size": 30811840, "index_size": 1762432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939116, "raw_average_key_size": 28, "raw_value_size": 108278650, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926397, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290039, "oldest_key_time": 0, "file_creation_time": 1732290039, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234349, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:43.629167 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234350.sst) /dev/shm/ToplingTemp/Topling-336247-LCeAND 2024/11/22-23:40:44.472231 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234350.sst): first pass time = 0.84's, 159.209'MB/sec 2024/11/22-23:40:46.015283 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234350.sst): old prealloc_size = 147639500, real_size = 33121104 2024/11/22-23:40:46.015855 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234350.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929008 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.47 avg-zval = 33.50 2024/11/22-23:40:46.017939 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234350.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:46.966256 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234350.sst): second pass time = 1.54's, 70.157'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.146'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3202.389'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 286.970'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 39728.216'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.157'MB/sec zip pipeline throughput = 70.157'MB/sec entries = 929008 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.47 avg-zval = 33.50 usrkeys = 929008 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.47 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929008 bytes = 3251536 } seq expand size = 6503056 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5156 value = 3.4768 dict = 1.00 all = 3.6055 } Zip/UnZip{ index = 0.0951 value = 0.2876 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 1.132240 GB avg = 0.122 KB (by entry num) total key len = 0.259592 GB avg = 0.028 KB total ukey len = 0.185423 GB avg = 0.020 KB total ukey num = 0.009271137 Billion total entry num = 0.009271137 Billion write speed all = 0.026111267 MB/sec (with seq num) write speed all = 0.024719828 MB/sec (without seq num) 2024/11/22-23:40:46.998728 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234350: 929008 keys, 37225846 bytes, temperature: kWarm 2024/11/22-23:40:46.998794 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290046998756, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234350, "file_size": 37225846, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3649396510, "table_properties": {"data_size": 31121728, "index_size": 1766912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26012224, "raw_average_key_size": 28, "raw_value_size": 108205552, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290043, "oldest_key_time": 0, "file_creation_time": 1732290043, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234350, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:47.036026 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234351.sst) /dev/shm/ToplingTemp/Topling-336247-rTDB3A 2024/11/22-23:40:47.939097 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234351.sst): first pass time = 0.90's, 148.630'MB/sec 2024/11/22-23:40:49.302471 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234351.sst): old prealloc_size = 147639500, real_size = 34212352 2024/11/22-23:40:49.303052 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234351.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931072 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0322 GB avg-val = 116.15 avg-zval = 34.59 2024/11/22-23:40:49.303617 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234351.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:50.164753 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234351.sst): second pass time = 1.36's, 79.361'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.283'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3157.915'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 284.037'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32926.966'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.361'MB/sec zip pipeline throughput = 79.361'MB/sec entries = 931072 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 34.59 usrkeys = 931072 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.03 avg-zval = 34.59 TagRS{ kinds = RS_Key0_TagN bytes = 145560 } TagArray{ size = 648729 bytes = 2108384 } seq expand size = 4541103 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5169 value = 3.3577 dict = 1.00 all = 3.5963 } Zip/UnZip{ index = 0.0951 value = 0.2978 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 1.244929 GB avg = 0.122 KB (by entry num) total key len = 0.285662 GB avg = 0.028 KB total ukey len = 0.204044 GB avg = 0.020 KB total ukey num = 0.010202209 Billion total entry num = 0.010202209 Billion write speed all = 0.028712713 MB/sec (with seq num) write speed all = 0.027181627 MB/sec (without seq num) 2024/11/22-23:40:50.198100 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234351: 931072 keys, 37321448 bytes, temperature: kWarm 2024/11/22-23:40:50.198170 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290050198138, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234351, "file_size": 37321448, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3649396426, "table_properties": {"data_size": 32208752, "index_size": 1770624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070016, "raw_average_key_size": 28, "raw_value_size": 108147816, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931072, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290047, "oldest_key_time": 0, "file_creation_time": 1732290047, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234351, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:50.235308 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234352.sst) /dev/shm/ToplingTemp/Topling-336247-5JRqyD 2024/11/22-23:40:51.061503 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234352.sst): first pass time = 0.83's, 162.461'MB/sec 2024/11/22-23:40:52.471255 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234352.sst): old prealloc_size = 147639500, real_size = 34978800 2024/11/22-23:40:52.471874 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234352.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081558 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.10 avg-zval = 30.19 2024/11/22-23:40:52.471920 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234352.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:53.097875 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234352.sst): second pass time = 1.41's, 73.789'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.058'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3442.035'MB/sec (index lex order gen) rebuild zvType time = 0.48's, 0.558'MB/sec write SST data time = 0.14's, 252.977'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.034'MB, throughput = 35258.261'MB/sec zip my value time = 1.41's, unzip length = 0.104'GB zip my value throughput = 73.789'MB/sec zip pipeline throughput = 73.789'MB/sec entries = 1081558 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.10 avg-zval = 30.19 usrkeys = 1081558 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.10 avg-zval = 30.19 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5161 value = 3.1832 dict = 1.00 all = 3.7540 } Zip/UnZip{ index = 0.0951 value = 0.3142 dict = 1.00 all = 0.2664 } ---------------------------- total value len = 1.348863 GB avg = 0.120 KB (by entry num) total key len = 0.315945 GB avg = 0.028 KB total ukey len = 0.225675 GB avg = 0.020 KB total ukey num = 0.011283767 Billion total entry num = 0.011283767 Billion write speed all = 0.031228819 MB/sec (with seq num) write speed all = 0.029535513 MB/sec (without seq num) 2024/11/22-23:40:53.132063 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234352: 1081558 keys, 35753176 bytes, temperature: kWarm 2024/11/22-23:40:53.132139 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290053132105, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234352, "file_size": 35753176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32651296, "index_size": 2056960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30283624, "raw_average_key_size": 28, "raw_value_size": 103934209, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081558, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290050, "oldest_key_time": 0, "file_creation_time": 1732290050, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234352, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:53.169918 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234353.sst) /dev/shm/ToplingTemp/Topling-336247-khXWOE 2024/11/22-23:40:53.997827 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234353.sst): first pass time = 0.83's, 162.125'MB/sec 2024/11/22-23:40:55.317982 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234353.sst): old prealloc_size = 147639500, real_size = 31708992 2024/11/22-23:40:55.318524 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234353.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919540 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0297 GB avg-val = 117.96 avg-zval = 32.32 2024/11/22-23:40:55.320743 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234353.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:56.185384 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234353.sst): second pass time = 1.32's, 82.234'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.460'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3386.803'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.314'MB/sec write SST data time = 0.13's, 267.810'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32494.864'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.234'MB/sec zip pipeline throughput = 82.234'MB/sec entries = 919540 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.96 avg-zval = 32.32 usrkeys = 919540 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.78 avg-zval = 32.32 TagRS{ kinds = RS_Key0_TagN bytes = 143760 } TagArray{ size = 895038 bytes = 3244528 } seq expand size = 6265266 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.09 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4451 value = 3.6500 dict = 1.00 all = 3.7322 } Zip/UnZip{ index = 0.0957 value = 0.2740 dict = 1.00 all = 0.2679 } ---------------------------- total value len = 1.463599 GB avg = 0.120 KB (by entry num) total key len = 0.341693 GB avg = 0.028 KB total ukey len = 0.244066 GB avg = 0.020 KB total ukey num = 0.012203307 Billion total entry num = 0.012203307 Billion write speed all = 0.033862068 MB/sec (with seq num) write speed all = 0.032030877 MB/sec (without seq num) 2024/11/22-23:40:56.219131 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234353: 919540 keys, 35961916 bytes, temperature: kWarm 2024/11/22-23:40:56.219194 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290056219162, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234353, "file_size": 35961916, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3604984147, "table_properties": {"data_size": 29718192, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25747120, "raw_average_key_size": 28, "raw_value_size": 108470696, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919540, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290053, "oldest_key_time": 0, "file_creation_time": 1732290053, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234353, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:56.257622 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234354.sst) /dev/shm/ToplingTemp/Topling-336247-LukqqC 2024/11/22-23:40:57.065059 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234354.sst): first pass time = 0.81's, 166.236'MB/sec 2024/11/22-23:40:58.382647 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234354.sst): old prealloc_size = 147639500, real_size = 34075856 2024/11/22-23:40:58.383238 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234354.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929584 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0321 GB avg-val = 116.38 avg-zval = 34.50 2024/11/22-23:40:58.385731 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234354.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:40:59.303263 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234354.sst): second pass time = 1.32's, 82.194'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.602'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3098.923'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 295.266'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33898.009'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.194'MB/sec zip pipeline throughput = 82.194'MB/sec entries = 929584 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.38 avg-zval = 34.50 usrkeys = 929584 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.38 avg-zval = 34.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929584 bytes = 3369760 } seq expand size = 6507088 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5103 value = 3.3731 dict = 1.00 all = 3.5045 } Zip/UnZip{ index = 0.0951 value = 0.2965 dict = 1.00 all = 0.2853 } ---------------------------- total value len = 1.578295 GB avg = 0.120 KB (by entry num) total key len = 0.367721 GB avg = 0.028 KB total ukey len = 0.262658 GB avg = 0.020 KB total ukey num = 0.013132891 Billion total entry num = 0.013132891 Billion write speed all = 0.036499527 MB/sec (with seq num) write speed all = 0.034528960 MB/sec (without seq num) 2024/11/22-23:40:59.340049 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234354: 929584 keys, 38298540 bytes, temperature: kWarm 2024/11/22-23:40:59.340112 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290059340080, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234354, "file_size": 38298540, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3649395319, "table_properties": {"data_size": 32074368, "index_size": 1768896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26028352, "raw_average_key_size": 28, "raw_value_size": 108189424, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929584, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290056, "oldest_key_time": 0, "file_creation_time": 1732290056, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234354, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:40:59.380231 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234355.sst) /dev/shm/ToplingTemp/Topling-336247-CWa1TE 2024/11/22-23:41:00.214922 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234355.sst): first pass time = 0.83's, 160.807'MB/sec 2024/11/22-23:41:01.554014 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234355.sst): old prealloc_size = 147639500, real_size = 32870720 2024/11/22-23:41:01.554530 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234355.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925686 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.99 avg-zval = 33.36 2024/11/22-23:41:01.557100 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234355.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:41:02.453114 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234355.sst): second pass time = 1.34's, 81.032'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 41.243'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3570.030'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.13's, 286.767'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31893.901'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.032'MB/sec zip pipeline throughput = 81.032'MB/sec entries = 925686 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.99 avg-zval = 33.36 usrkeys = 925686 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.99 avg-zval = 33.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925686 bytes = 3355632 } seq expand size = 6479802 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5138 value = 3.5073 dict = 1.00 all = 3.6196 } Zip/UnZip{ index = 0.0951 value = 0.2851 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 1.693074 GB avg = 0.120 KB (by entry num) total key len = 0.393640 GB avg = 0.028 KB total ukey len = 0.281172 GB avg = 0.020 KB total ukey num = 0.014058577 Billion total entry num = 0.014058577 Billion write speed all = 0.039136141 MB/sec (with seq num) write speed all = 0.037026802 MB/sec (without seq num) 2024/11/22-23:41:02.486205 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234355: 925686 keys, 37081094 bytes, temperature: kWarm 2024/11/22-23:41:02.486269 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290062486237, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234355, "file_size": 37081094, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3649395436, "table_properties": {"data_size": 30878192, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919208, "raw_average_key_size": 28, "raw_value_size": 108298560, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925686, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290059, "oldest_key_time": 0, "file_creation_time": 1732290059, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234355, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:41:02.523198 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234356.sst) /dev/shm/ToplingTemp/Topling-336247-5U6d1B 2024/11/22-23:41:02.621445 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234356.sst): first pass time = 0.10's, 224.975'MB/sec 2024/11/22-23:41:02.819573 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234356.sst): old prealloc_size = 147639500, real_size = 6122464 2024/11/22-23:41:02.819709 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234356.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 154280 prefix = 4 raw-key = 0.0025 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.0178 GB zip-val = 0.0058 GB avg-val = 115.20 avg-zval = 37.50 2024/11/22-23:41:02.819872 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234356.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.022 GB 2024/11/22-23:41:02.965840 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00/234356.sst): second pass time = 0.20's, 89.976'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 46.316'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2742.236'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.310'MB/sec write SST data time = 0.02's, 314.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.187'MB, throughput = 6210.313'MB/sec zip my value time = 0.20's, unzip length = 0.018'GB zip my value throughput = 89.976'MB/sec zip pipeline throughput = 89.976'MB/sec entries = 154280 avg-key = 28.00 avg-zkey = 1.94 avg-val = 115.20 avg-zval = 37.50 usrkeys = 154280 avg-key = 20.00 avg-zkey = 1.94 avg-val = 122.20 avg-zval = 37.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 154280 bytes = 501424 } seq expand size = 1079960 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0031 GB value = 0.0178 GB dict = 0.19 MB all = 0.0221 GB } __ZipSize{ index = 0.0003 GB value = 0.0058 GB dict = 0.19 MB all = 0.0068 GB } UnZip/Zip{ index = 10.3327 value = 3.0723 dict = 1.00 all = 3.2617 } Zip/UnZip{ index = 0.0968 value = 0.3255 dict = 1.00 all = 0.3066 } ---------------------------- total value len = 1.711927 GB avg = 0.120 KB (by entry num) total key len = 0.397960 GB avg = 0.028 KB total ukey len = 0.284257 GB avg = 0.020 KB total ukey num = 0.014212857 Billion total entry num = 0.014212857 Billion write speed all = 0.039570370 MB/sec (with seq num) write speed all = 0.037437903 MB/sec (without seq num) 2024/11/22-23:41:02.980157 140024902895168 [compaction_job.cc:1948] [default] [JOB 14231] Generated table #234356: 154280 keys, 6773587 bytes, temperature: kWarm 2024/11/22-23:41:02.980218 140024902895168 EVENT_LOG_v1 {"time_micros": 1732290062980187, "cf_name": "default", "job": 14231, "event": "table_file_creation", "file_number": 234356, "file_size": 6773587, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3606850670, "largest_seqno": 3648158029, "table_properties": {"data_size": 5785104, "index_size": 298624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4319840, "raw_average_key_size": 28, "raw_value_size": 17773292, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 154280, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290062, "oldest_key_time": 0, "file_creation_time": 1732290062, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234356, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:41:02.994205 140024902895168 [compaction_job.cc:703] job-14231: subcompact[0], size: 0.566104 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14231/att-00 [234341,234342,234343,234344,234345,234346,234347,234348,234349,234350,234351,234352,234353,234354,234355,234356] 2024/11/22-23:41:03.096720 140024902895168 (Original Log Time 2024/11/22-23:40:08.439668) EVENT_LOG_v1 {"time_micros": 1732290008439629, "job": 14231, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [234337], "files_L3": [234289, 234290, 234291, 234292, 234293, 234294, 234295, 234303, 234304, 234305, 234306, 234307, 234308, 234309, 234310], "score": 0.998925, "input_data_size": 588911150, "oldest_snapshot_seqno": 3649420582} 2024/11/22-23:41:03.096724 140024902895168 (Original Log Time 2024/11/22-23:41:03.095875) [compaction_job.cc:2022] [default] [JOB 14231] Compacted 1@2 + 15@3 files to L3 => 566104073 bytes 2024/11/22-23:41:03.096726 140024902895168 (Original Log Time 2024/11/22-23:41:03.096657) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.8 rd, 10.4 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.1, 534.5 +0.0 blob) out(539.9 +0.0 blob), read-write-amplify(40.7) write-amplify(19.9) OK, records in: 14293201, records dropped: 80344 output_compression: Snappy 2024/11/22-23:41:03.096732 140024902895168 (Original Log Time 2024/11/22-23:41:03.096700) EVENT_LOG_v1 {"time_micros": 1732290063096679, "cf": "default", "job": 14231, "event": "compaction_finished", "compaction_time_micros": 54554546, "compaction_time_cpu_micros": 24003574, "output_level": 3, "num_output_files": 16, "total_output_size": 566104073, "num_input_records": 14293201, "num_output_records": 14212857, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 388386725, "file_range_sync_nanos": 0, "file_fsync_nanos": 16236, "file_prepare_write_nanos": 43746, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-23:41:03.235453 140024902895168 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732290008,"executesMs":54800,"compactionJobId":14231,"attempt":0,"compactionInputRawBytes":2046769051,"compactionInputZipBytes":588911150,"compactionOutputRawBytes":0,"compactionOutputZipBytes":566104073,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-2ab12035f3e8a355653dac0a50ccc2cd-3166d7c52fc78c87-00"}}, response = 2024/11/22-23:41:03.235532 140024902895168 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-14231/att-00: olev 3, work 54.801 s, result 5.174 ms, install 0.877 ms, input{raw 1.906 GiB zip 561.629 MiB}