2024/11/20-21:37:41.907498 140008937477696 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00: fork to child time = 0.008317 sec 2024/11/20-21:37:41.908086 140008937477696 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00 2024/11/20-21:37:41.914174 140008937477696 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00 2024/11/20-21:37:41.915757 140008937477696 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-21:37:41.915923 140008937477696 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-21:37:41.961325 140008937477696 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-14817/att-00/MANIFEST-049091 2024/11/20-21:37:41.976841 140008937477696 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-14817/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 74256, last_sequence is 1176745866, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 74128 2024/11/20-21:37:41.976848 140008937477696 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-21:37:41.976922 140008937477696 [version_set.cc:5724] Creating manifest 74256 2024/11/20-21:37:42.008143 140008937477696 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-21:37:42.026682 140008937477696 [compaction_job.cc:2437] [default] [JOB 14817] Compacting 1@2 + 21@3 files to L3, score 0.93, subcompactions 1 : 1 2024/11/20-21:37:42.026694 140008937477696 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 11723 Base level 2, inputs: [74253(54MB)], [74219(41MB) 74220(36MB) 74221(37MB) 74222(38MB) 74223(37MB) 74224(36MB) 74225(40MB) 74226(37MB) 74227(37MB) 74228(37MB) 74229(37MB) 74230(37MB) 74231(37MB) 74232(37MB) 74233(6173KB) 74133(38MB) 74134(37MB) 74135(37MB) 74191(37MB) 74192(37MB) 74193(38MB)] 2024/11/20-21:37:42.036454 140008937477696 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-21:37:42.036472 140008937477696 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-21:37:42.036487 140008937477696 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-21:37:42.036493 140008937477696 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-21:37:42.036501 140008937477696 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-21:37:42.851413 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074257.sst) /dev/shm/ToplingTemp/Topling-1024810-p3lw7w 2024/11/20-21:37:43.382159 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074257.sst): first pass time = 0.53's, 252.903'MB/sec 2024/11/20-21:37:45.621099 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074257.sst): old prealloc_size = 147639500, real_size = 39018352 2024/11/20-21:37:45.621656 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074257.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923267 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0370 GB avg-val = 117.37 avg-zval = 40.07 2024/11/20-21:37:45.623147 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074257.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:37:46.501191 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074257.sst): second pass time = 2.24's, 48.418'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 48.985'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3353.646'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.310'MB/sec write SST data time = 0.14's, 318.841'MB/sec dict compress time = 4338989792.57's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 28287.095'MB/sec zip my value time = 2.24's, unzip length = 0.108'GB zip my value throughput = 48.418'MB/sec zip pipeline throughput = 135.993'MB/sec entries = 923267 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.37 avg-zval = 40.07 usrkeys = 923267 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.37 avg-zval = 40.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923267 bytes = 3231456 } seq expand size = 6462869 multi value expand size = 0 cnt WriteAppend = 39 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0370 GB dict = 1.08 MB all = 0.0431 GB } UnZip/Zip{ index = 10.3076 value = 2.9291 dict = 1.00 all = 3.1133 } Zip/UnZip{ index = 0.0970 value = 0.3414 dict = 1.00 all = 0.3212 } ---------------------------- total value len = 563.180561 GB avg = 0.331 KB (by entry num) total key len = 44.340258 GB avg = 0.026 KB total ukey len = 32.812607 GB avg = 0.014 KB total ukey num = 2.321937843 Billion total entry num = 1.701439865 Billion write speed all = 47.299408252 MB/sec (with seq num) write speed all = 46.239663845 MB/sec (without seq num) 2024/11/20-21:37:46.553703 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74257: 923267 keys, 43111128 bytes, temperature: kWarm 2024/11/20-21:37:46.553788 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109866553747, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74257, "file_size": 43111128, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 939479839, "largest_seqno": 1175941780, "table_properties": {"data_size": 36995936, "index_size": 1791424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25851476, "raw_average_key_size": 28, "raw_value_size": 108366306, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923267, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109862, "oldest_key_time": 0, "file_creation_time": 1732109862, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74257, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:37:46.591775 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074258.sst) /dev/shm/ToplingTemp/Topling-1024810-ihyyTx 2024/11/20-21:37:47.418079 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074258.sst): first pass time = 0.83's, 162.438'MB/sec 2024/11/20-21:37:48.824224 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074258.sst): old prealloc_size = 147639500, real_size = 33999184 2024/11/20-21:37:48.824803 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074258.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921492 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.65 avg-zval = 34.74 2024/11/20-21:37:48.825646 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074258.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:37:49.793299 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074258.sst): second pass time = 1.41's, 77.142'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 39.300'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3142.359'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.14's, 266.845'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 34105.684'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 77.142'MB/sec zip pipeline throughput = 77.142'MB/sec entries = 921492 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.65 avg-zval = 34.74 usrkeys = 921492 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.65 avg-zval = 34.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921492 bytes = 3225232 } seq expand size = 6450444 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5139 value = 3.3863 dict = 1.00 all = 3.5243 } Zip/UnZip{ index = 0.0951 value = 0.2953 dict = 1.00 all = 0.2837 } ---------------------------- total value len = 563.295428 GB avg = 0.331 KB (by entry num) total key len = 44.366060 GB avg = 0.026 KB total ukey len = 32.831037 GB avg = 0.014 KB total ukey num = 2.322859335 Billion total entry num = 1.702361357 Billion write speed all = 47.298237081 MB/sec (with seq num) write speed all = 46.238190424 MB/sec (without seq num) 2024/11/20-21:37:49.842343 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74258: 921492 keys, 38083562 bytes, temperature: kWarm 2024/11/20-21:37:49.842410 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109869842379, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74258, "file_size": 38083562, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 952414140, "largest_seqno": 1175939282, "table_properties": {"data_size": 32015744, "index_size": 1752896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25801776, "raw_average_key_size": 28, "raw_value_size": 108416036, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921492, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109866, "oldest_key_time": 0, "file_creation_time": 1732109866, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74258, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:37:49.880896 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074259.sst) /dev/shm/ToplingTemp/Topling-1024810-cMFhSy 2024/11/20-21:37:50.801961 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074259.sst): first pass time = 0.92's, 145.726'MB/sec 2024/11/20-21:37:52.258893 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074259.sst): old prealloc_size = 147639500, real_size = 35276512 2024/11/20-21:37:52.259520 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074259.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926084 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 116.93 avg-zval = 35.93 2024/11/20-21:37:52.260330 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074259.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:37:53.305881 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074259.sst): second pass time = 1.46's, 74.359'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 37.496'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2926.128'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.253'MB/sec write SST data time = 0.13's, 299.642'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 25414.219'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.359'MB/sec zip pipeline throughput = 131.540'MB/sec entries = 926084 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.93 avg-zval = 35.93 usrkeys = 926084 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.93 avg-zval = 35.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926084 bytes = 3125552 } seq expand size = 6482588 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4424 value = 3.2547 dict = 1.00 all = 3.4190 } Zip/UnZip{ index = 0.0958 value = 0.3072 dict = 1.00 all = 0.2925 } ---------------------------- total value len = 563.410198 GB avg = 0.331 KB (by entry num) total key len = 44.391990 GB avg = 0.026 KB total ukey len = 32.849559 GB avg = 0.014 KB total ukey num = 2.323785419 Billion total entry num = 1.703287441 Billion write speed all = 47.296257572 MB/sec (with seq num) write speed all = 46.235924152 MB/sec (without seq num) 2024/11/20-21:37:53.352931 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74259: 926084 keys, 39256974 bytes, temperature: kWarm 2024/11/20-21:37:53.352997 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109873352966, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74259, "file_size": 39256974, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1069697601, "largest_seqno": 1175941707, "table_properties": {"data_size": 33271120, "index_size": 1773696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25930352, "raw_average_key_size": 28, "raw_value_size": 108287404, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926084, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109869, "oldest_key_time": 0, "file_creation_time": 1732109869, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74259, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:37:53.389363 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074260.sst) /dev/shm/ToplingTemp/Topling-1024810-mWeTBu 2024/11/20-21:37:54.709135 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074260.sst): first pass time = 1.32's, 101.700'MB/sec 2024/11/20-21:37:55.922685 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074260.sst): old prealloc_size = 147639500, real_size = 36449104 2024/11/20-21:37:55.923384 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074260.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929633 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1082 GB zip-val = 0.0344 GB avg-val = 116.38 avg-zval = 36.99 2024/11/20-21:37:55.924285 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074260.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:37:57.008127 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074260.sst): second pass time = 1.21's, 89.225'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 37.356'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2715.197'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.272'MB/sec write SST data time = 0.23's, 175.137'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32139.743'MB/sec zip my value time = 1.21's, unzip length = 0.108'GB zip my value throughput = 89.225'MB/sec zip pipeline throughput = 89.225'MB/sec entries = 929633 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.38 avg-zval = 36.99 usrkeys = 929633 avg-key = 20.00 avg-zkey = 1.97 avg-val = 122.81 avg-zval = 36.99 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 854467 bytes = 3097456 } seq expand size = 5981269 multi value expand size = 0 cnt WriteAppend = 86 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.1780 value = 3.1459 dict = 1.00 all = 3.3101 } Zip/UnZip{ index = 0.0983 value = 0.3179 dict = 1.00 all = 0.3021 } ---------------------------- total value len = 563.524367 GB avg = 0.331 KB (by entry num) total key len = 44.418020 GB avg = 0.026 KB total ukey len = 32.868151 GB avg = 0.014 KB total ukey num = 2.324715052 Billion total entry num = 1.704217074 Billion write speed all = 47.293542322 MB/sec (with seq num) write speed all = 46.232935738 MB/sec (without seq num) 2024/11/20-21:37:57.061884 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74260: 929633 keys, 40548068 bytes, temperature: kWarm 2024/11/20-21:37:57.061965 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109877061920, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74260, "file_size": 40548068, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1175942564, "table_properties": {"data_size": 34389760, "index_size": 1826752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26029724, "raw_average_key_size": 28, "raw_value_size": 108188102, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929633, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109873, "oldest_key_time": 0, "file_creation_time": 1732109873, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74260, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:37:57.142397 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074261.sst) /dev/shm/ToplingTemp/Topling-1024810-PvDWVu 2024/11/20-21:37:58.256002 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074261.sst): first pass time = 1.11's, 120.532'MB/sec 2024/11/20-21:37:59.864699 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074261.sst): old prealloc_size = 147639500, real_size = 36400288 2024/11/20-21:37:59.865279 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074261.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926416 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0343 GB avg-val = 116.88 avg-zval = 37.08 2024/11/20-21:37:59.865877 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074261.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:00.995900 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074261.sst): second pass time = 1.61's, 67.341'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 34.909'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3266.447'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.268'MB/sec write SST data time = 0.27's, 147.531'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 19309.337'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.341'MB/sec zip pipeline throughput = 149.205'MB/sec entries = 926416 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.88 avg-zval = 37.08 usrkeys = 926416 avg-key = 20.00 avg-zkey = 1.97 avg-val = 121.54 avg-zval = 37.08 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 617447 bytes = 2083904 } seq expand size = 4322129 multi value expand size = 0 cnt WriteAppend = 52 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.1766 value = 3.1524 dict = 1.00 all = 3.3990 } Zip/UnZip{ index = 0.0983 value = 0.3172 dict = 1.00 all = 0.2942 } ---------------------------- total value len = 563.636968 GB avg = 0.331 KB (by entry num) total key len = 44.443959 GB avg = 0.026 KB total ukey len = 32.886680 GB avg = 0.014 KB total ukey num = 2.325641468 Billion total entry num = 1.705143490 Billion write speed all = 47.289649663 MB/sec (with seq num) write speed all = 46.228795626 MB/sec (without seq num) 2024/11/20-21:38:01.050823 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74261: 926416 keys, 39487452 bytes, temperature: kWarm 2024/11/20-21:38:01.050913 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109881050867, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74261, "file_size": 39487452, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 978567417, "table_properties": {"data_size": 34347856, "index_size": 1820672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939648, "raw_average_key_size": 28, "raw_value_size": 108278212, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926416, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109877, "oldest_key_time": 0, "file_creation_time": 1732109877, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74261, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:01.119982 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074262.sst) /dev/shm/ToplingTemp/Topling-1024810-Iqg3tu 2024/11/20-21:38:02.000220 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074262.sst): first pass time = 0.88's, 152.498'MB/sec 2024/11/20-21:38:03.365430 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074262.sst): old prealloc_size = 147639500, real_size = 36510624 2024/11/20-21:38:03.365999 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074262.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927066 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.17 2024/11/20-21:38:03.366207 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074262.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:04.245985 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074262.sst): second pass time = 1.36's, 79.348'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.871'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3390.115'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.307'MB/sec write SST data time = 0.13's, 300.558'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18924.605'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.348'MB/sec zip pipeline throughput = 79.348'MB/sec entries = 927066 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.78 avg-zval = 37.17 usrkeys = 927066 avg-key = 20.00 avg-zkey = 1.97 avg-val = 117.30 avg-zval = 37.17 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 69826 bytes = 235680 } seq expand size = 488782 multi value expand size = 0 cnt WriteAppend = 43 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.1745 value = 3.1419 dict = 1.00 all = 3.5555 } Zip/UnZip{ index = 0.0983 value = 0.3183 dict = 1.00 all = 0.2813 } ---------------------------- total value len = 563.745716 GB avg = 0.330 KB (by entry num) total key len = 44.469917 GB avg = 0.026 KB total ukey len = 32.905221 GB avg = 0.014 KB total ukey num = 2.326568534 Billion total entry num = 1.706070556 Billion write speed all = 47.288173196 MB/sec (with seq num) write speed all = 46.227010601 MB/sec (without seq num) 2024/11/20-21:38:04.293816 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74262: 927066 keys, 37749465 bytes, temperature: kWarm 2024/11/20-21:38:04.293882 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109884293850, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74262, "file_size": 37749465, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 972340122, "table_properties": {"data_size": 34456336, "index_size": 1822336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957848, "raw_average_key_size": 28, "raw_value_size": 108259884, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927066, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109881, "oldest_key_time": 0, "file_creation_time": 1732109881, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74262, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:04.331977 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074263.sst) /dev/shm/ToplingTemp/Topling-1024810-ptb4Ou 2024/11/20-21:38:05.163489 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074263.sst): first pass time = 0.83's, 161.420'MB/sec 2024/11/20-21:38:06.407794 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074263.sst): old prealloc_size = 147639500, real_size = 38485456 2024/11/20-21:38:06.408485 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074263.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925297 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0365 GB avg-val = 117.05 avg-zval = 39.43 2024/11/20-21:38:06.409194 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074263.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:07.411415 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074263.sst): second pass time = 1.24's, 87.097'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 41.853'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2637.921'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.265'MB/sec write SST data time = 0.13's, 322.779'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 19513.229'MB/sec zip my value time = 1.24's, unzip length = 0.108'GB zip my value throughput = 87.097'MB/sec zip pipeline throughput = 87.097'MB/sec entries = 925297 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.05 avg-zval = 39.43 usrkeys = 925297 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.30 avg-zval = 39.43 TagRS{ kinds = RS_Key0_TagN bytes = 144680 } TagArray{ size = 693804 bytes = 2515056 } seq expand size = 4856628 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0365 GB dict = 1.08 MB all = 0.0420 GB } UnZip/Zip{ index = 10.4467 value = 2.9688 dict = 1.00 all = 3.1954 } Zip/UnZip{ index = 0.0957 value = 0.3368 dict = 1.00 all = 0.3130 } ---------------------------- total value len = 563.858882 GB avg = 0.330 KB (by entry num) total key len = 44.495825 GB avg = 0.026 KB total ukey len = 32.923727 GB avg = 0.014 KB total ukey num = 2.327493831 Billion total entry num = 1.706995853 Billion write speed all = 47.287348244 MB/sec (with seq num) write speed all = 46.225871361 MB/sec (without seq num) 2024/11/20-21:38:07.462800 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74263: 925297 keys, 42003900 bytes, temperature: kWarm 2024/11/20-21:38:07.462869 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109887462836, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74263, "file_size": 42003900, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1175941334, "table_properties": {"data_size": 36482496, "index_size": 1771456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25908316, "raw_average_key_size": 28, "raw_value_size": 108309414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925297, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109884, "oldest_key_time": 0, "file_creation_time": 1732109884, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74263, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:07.497678 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074264.sst) /dev/shm/ToplingTemp/Topling-1024810-xs2Arv 2024/11/20-21:38:08.303332 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074264.sst): first pass time = 0.81's, 166.603'MB/sec 2024/11/20-21:38:09.585386 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074264.sst): old prealloc_size = 147639500, real_size = 36971792 2024/11/20-21:38:09.586170 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074264.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923793 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0350 GB avg-val = 117.29 avg-zval = 37.87 2024/11/20-21:38:09.586815 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074264.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:10.547325 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074264.sst): second pass time = 1.28's, 84.581'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 41.709'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2323.597'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.16's, 244.054'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31159.522'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.581'MB/sec zip pipeline throughput = 84.581'MB/sec entries = 923793 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.29 avg-zval = 37.87 usrkeys = 923793 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.93 avg-zval = 37.87 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 612426 bytes = 2143504 } seq expand size = 4286982 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0350 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.5126 value = 3.0972 dict = 1.00 all = 3.3454 } Zip/UnZip{ index = 0.0951 value = 0.3229 dict = 1.00 all = 0.2989 } ---------------------------- total value len = 563.971521 GB avg = 0.330 KB (by entry num) total key len = 44.521692 GB avg = 0.026 KB total ukey len = 32.942203 GB avg = 0.014 KB total ukey num = 2.328417624 Billion total entry num = 1.707919646 Billion write speed all = 47.286587918 MB/sec (with seq num) write speed all = 46.224795401 MB/sec (without seq num) 2024/11/20-21:38:10.595526 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74264: 923793 keys, 40120678 bytes, temperature: kWarm 2024/11/20-21:38:10.595600 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109890595568, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74264, "file_size": 40120678, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1175942350, "table_properties": {"data_size": 34983168, "index_size": 1757504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25866204, "raw_average_key_size": 28, "raw_value_size": 108351610, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923793, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109887, "oldest_key_time": 0, "file_creation_time": 1732109887, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74264, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:10.630573 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074265.sst) /dev/shm/ToplingTemp/Topling-1024810-mVHE0x 2024/11/20-21:38:11.510601 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074265.sst): first pass time = 0.88's, 152.522'MB/sec 2024/11/20-21:38:12.638200 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074265.sst): old prealloc_size = 147639500, real_size = 35501648 2024/11/20-21:38:12.638816 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074265.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921772 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0335 GB avg-val = 117.61 avg-zval = 36.36 2024/11/20-21:38:12.639594 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074265.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:13.474208 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074265.sst): second pass time = 1.13's, 96.211'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.350'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2958.714'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.14's, 289.897'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 19559.430'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 96.211'MB/sec zip pipeline throughput = 96.211'MB/sec entries = 921772 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 36.36 usrkeys = 921772 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.29 avg-zval = 36.36 TagRS{ kinds = RS_Key0_TagN bytes = 144120 } TagArray{ size = 879634 bytes = 3078736 } seq expand size = 6157438 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5129 value = 3.2344 dict = 1.00 all = 3.3907 } Zip/UnZip{ index = 0.0951 value = 0.3092 dict = 1.00 all = 0.2949 } ---------------------------- total value len = 564.086087 GB avg = 0.330 KB (by entry num) total key len = 44.547501 GB avg = 0.026 KB total ukey len = 32.960638 GB avg = 0.014 KB total ukey num = 2.329339396 Billion total entry num = 1.708841418 Billion write speed all = 47.286741196 MB/sec (with seq num) write speed all = 46.224617206 MB/sec (without seq num) 2024/11/20-21:38:13.526572 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74265: 921772 keys, 39583708 bytes, temperature: kWarm 2024/11/20-21:38:13.526630 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109893526601, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74265, "file_size": 39583708, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1175942375, "table_properties": {"data_size": 33517440, "index_size": 1753600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25809616, "raw_average_key_size": 28, "raw_value_size": 108408176, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921772, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109890, "oldest_key_time": 0, "file_creation_time": 1732109890, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74265, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:13.563912 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074266.sst) /dev/shm/ToplingTemp/Topling-1024810-d3qXby 2024/11/20-21:38:14.376191 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074266.sst): first pass time = 0.81's, 165.243'MB/sec 2024/11/20-21:38:15.494224 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074266.sst): old prealloc_size = 147639500, real_size = 35480656 2024/11/20-21:38:15.494801 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074266.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922757 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0335 GB avg-val = 117.45 avg-zval = 36.30 2024/11/20-21:38:15.495623 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074266.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:16.336753 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074266.sst): second pass time = 1.12's, 97.012'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.962'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3184.422'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.14's, 287.161'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31371.021'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 97.012'MB/sec zip pipeline throughput = 97.012'MB/sec entries = 922757 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.45 avg-zval = 36.30 usrkeys = 922757 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.38 avg-zval = 36.30 TagRS{ kinds = RS_Key0_TagN bytes = 144280 } TagArray{ size = 913293 bytes = 3082384 } seq expand size = 6393051 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5115 value = 3.2358 dict = 1.00 all = 3.3923 } Zip/UnZip{ index = 0.0951 value = 0.3090 dict = 1.00 all = 0.2948 } ---------------------------- total value len = 564.200860 GB avg = 0.330 KB (by entry num) total key len = 44.573338 GB avg = 0.026 KB total ukey len = 32.979093 GB avg = 0.014 KB total ukey num = 2.330262153 Billion total entry num = 1.709764175 Billion write speed all = 47.287149029 MB/sec (with seq num) write speed all = 46.224687795 MB/sec (without seq num) 2024/11/20-21:38:16.387667 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74266: 922757 keys, 39565658 bytes, temperature: kWarm 2024/11/20-21:38:16.387726 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109896387696, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74266, "file_size": 39565658, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1175942409, "table_properties": {"data_size": 33494080, "index_size": 1755712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25837196, "raw_average_key_size": 28, "raw_value_size": 108380562, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922757, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109893, "oldest_key_time": 0, "file_creation_time": 1732109893, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74266, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:16.425175 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074267.sst) /dev/shm/ToplingTemp/Topling-1024810-77urkv 2024/11/20-21:38:17.237375 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074267.sst): first pass time = 0.81's, 165.259'MB/sec 2024/11/20-21:38:18.347824 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074267.sst): old prealloc_size = 147639500, real_size = 35497376 2024/11/20-21:38:18.348425 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074267.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922990 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0335 GB avg-val = 117.42 avg-zval = 36.31 2024/11/20-21:38:18.349186 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074267.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:19.182303 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074267.sst): second pass time = 1.11's, 97.668'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.301'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3038.869'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 300.188'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 19155.988'MB/sec zip my value time = 1.11's, unzip length = 0.108'GB zip my value throughput = 97.668'MB/sec zip pipeline throughput = 97.668'MB/sec entries = 922990 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 36.31 usrkeys = 922990 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.42 avg-zval = 36.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922990 bytes = 3115104 } seq expand size = 6460930 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5134 value = 3.2340 dict = 1.00 all = 3.4005 } Zip/UnZip{ index = 0.0951 value = 0.3092 dict = 1.00 all = 0.2941 } ---------------------------- total value len = 564.315695 GB avg = 0.330 KB (by entry num) total key len = 44.599182 GB avg = 0.026 KB total ukey len = 32.997553 GB avg = 0.014 KB total ukey num = 2.331185143 Billion total entry num = 1.710687165 Billion write speed all = 47.287624362 MB/sec (with seq num) write speed all = 46.224824486 MB/sec (without seq num) 2024/11/20-21:38:19.229638 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74267: 922990 keys, 39470424 bytes, temperature: kWarm 2024/11/20-21:38:19.229702 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109899229671, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74267, "file_size": 39470424, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1063676615, "largest_seqno": 1175942454, "table_properties": {"data_size": 33510608, "index_size": 1755840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25843720, "raw_average_key_size": 28, "raw_value_size": 108374116, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922990, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109896, "oldest_key_time": 0, "file_creation_time": 1732109896, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74267, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:19.263362 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074268.sst) /dev/shm/ToplingTemp/Topling-1024810-LABm7u 2024/11/20-21:38:20.076089 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074268.sst): first pass time = 0.81's, 165.151'MB/sec 2024/11/20-21:38:21.187216 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074268.sst): old prealloc_size = 147639500, real_size = 35609776 2024/11/20-21:38:21.187796 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074268.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922975 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0336 GB avg-val = 117.42 avg-zval = 36.43 2024/11/20-21:38:21.188582 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074268.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:22.021172 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074268.sst): second pass time = 1.11's, 97.603'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.470'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3139.392'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 303.159'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 19330.941'MB/sec zip my value time = 1.11's, unzip length = 0.108'GB zip my value throughput = 97.603'MB/sec zip pipeline throughput = 97.603'MB/sec entries = 922975 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 36.43 usrkeys = 922975 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.42 avg-zval = 36.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922975 bytes = 3115056 } seq expand size = 6460825 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5151 value = 3.2232 dict = 1.00 all = 3.3905 } Zip/UnZip{ index = 0.0951 value = 0.3103 dict = 1.00 all = 0.2949 } ---------------------------- total value len = 564.430530 GB avg = 0.330 KB (by entry num) total key len = 44.625026 GB avg = 0.026 KB total ukey len = 33.016013 GB avg = 0.014 KB total ukey num = 2.332108118 Billion total entry num = 1.711610140 Billion write speed all = 47.288124013 MB/sec (with seq num) write speed all = 46.224985102 MB/sec (without seq num) 2024/11/20-21:38:22.069641 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74268: 922975 keys, 39586468 bytes, temperature: kWarm 2024/11/20-21:38:22.069703 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109902069672, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74268, "file_size": 39586468, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1063678836, "largest_seqno": 1175942469, "table_properties": {"data_size": 33623328, "index_size": 1755520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25843300, "raw_average_key_size": 28, "raw_value_size": 108374478, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922975, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109899, "oldest_key_time": 0, "file_creation_time": 1732109899, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74268, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:22.108709 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074269.sst) /dev/shm/ToplingTemp/Topling-1024810-AEUhQv 2024/11/20-21:38:22.922781 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074269.sst): first pass time = 0.81's, 164.879'MB/sec 2024/11/20-21:38:24.057028 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074269.sst): old prealloc_size = 147639500, real_size = 35504640 2024/11/20-21:38:24.057630 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074269.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923007 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0335 GB avg-val = 117.41 avg-zval = 36.31 2024/11/20-21:38:24.058382 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074269.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:24.889180 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074269.sst): second pass time = 1.13's, 95.612'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.442'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3162.261'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 305.495'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34210.047'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 95.612'MB/sec zip pipeline throughput = 95.612'MB/sec entries = 923007 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.41 avg-zval = 36.31 usrkeys = 923007 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.41 avg-zval = 36.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923007 bytes = 3115168 } seq expand size = 6461049 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5139 value = 3.2333 dict = 1.00 all = 3.3998 } Zip/UnZip{ index = 0.0951 value = 0.3093 dict = 1.00 all = 0.2941 } ---------------------------- total value len = 564.545365 GB avg = 0.330 KB (by entry num) total key len = 44.650870 GB avg = 0.026 KB total ukey len = 33.034473 GB avg = 0.014 KB total ukey num = 2.333031125 Billion total entry num = 1.712533147 Billion write speed all = 47.288516481 MB/sec (with seq num) write speed all = 46.225041071 MB/sec (without seq num) 2024/11/20-21:38:24.936872 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74269: 923007 keys, 39477644 bytes, temperature: kWarm 2024/11/20-21:38:24.936934 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109904936904, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74269, "file_size": 39477644, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1063678877, "largest_seqno": 1175942528, "table_properties": {"data_size": 33517936, "index_size": 1755776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844196, "raw_average_key_size": 28, "raw_value_size": 108373578, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923007, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109902, "oldest_key_time": 0, "file_creation_time": 1732109902, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74269, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:24.970444 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074270.sst) /dev/shm/ToplingTemp/Topling-1024810-IfnOAw 2024/11/20-21:38:25.784161 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074270.sst): first pass time = 0.81's, 164.950'MB/sec 2024/11/20-21:38:26.933715 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074270.sst): old prealloc_size = 147639500, real_size = 35489808 2024/11/20-21:38:26.934316 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074270.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922951 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0335 GB avg-val = 117.42 avg-zval = 36.30 2024/11/20-21:38:26.935070 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074270.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:27.766218 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074270.sst): second pass time = 1.15's, 94.347'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.404'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3026.119'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 305.315'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 17676.391'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 94.347'MB/sec zip pipeline throughput = 94.347'MB/sec entries = 922951 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 36.30 usrkeys = 922951 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.42 avg-zval = 36.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922951 bytes = 3114976 } seq expand size = 6460657 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5110 value = 3.2348 dict = 1.00 all = 3.4011 } Zip/UnZip{ index = 0.0951 value = 0.3091 dict = 1.00 all = 0.2940 } ---------------------------- total value len = 564.660201 GB avg = 0.330 KB (by entry num) total key len = 44.676712 GB avg = 0.026 KB total ukey len = 33.052932 GB avg = 0.014 KB total ukey num = 2.333954076 Billion total entry num = 1.713456098 Billion write speed all = 47.288875617 MB/sec (with seq num) write speed all = 46.225064638 MB/sec (without seq num) 2024/11/20-21:38:27.813040 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74270: 922951 keys, 39462588 bytes, temperature: kWarm 2024/11/20-21:38:27.813114 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109907813082, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74270, "file_size": 39462588, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1062170413, "largest_seqno": 1175941931, "table_properties": {"data_size": 33502720, "index_size": 1756160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842628, "raw_average_key_size": 28, "raw_value_size": 108375106, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922951, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109904, "oldest_key_time": 0, "file_creation_time": 1732109904, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74270, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:27.846887 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074271.sst) /dev/shm/ToplingTemp/Topling-1024810-KljKeu 2024/11/20-21:38:28.728509 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074271.sst): first pass time = 0.88's, 152.245'MB/sec 2024/11/20-21:38:29.899442 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074271.sst): old prealloc_size = 147639500, real_size = 36133472 2024/11/20-21:38:29.899999 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074271.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924813 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0341 GB avg-val = 117.13 avg-zval = 36.91 2024/11/20-21:38:29.900778 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074271.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:30.728874 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074271.sst): second pass time = 1.17's, 92.573'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.443'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3320.850'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 304.389'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30210.856'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.573'MB/sec zip pipeline throughput = 92.573'MB/sec entries = 924813 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.13 avg-zval = 36.91 usrkeys = 924813 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.48 avg-zval = 36.91 TagRS{ kinds = RS_Key0_TagN bytes = 144600 } TagArray{ size = 839011 bytes = 3041424 } seq expand size = 5873077 multi value expand size = 0 cnt WriteAppend = 20 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.4462 value = 3.1737 dict = 1.00 all = 3.3405 } Zip/UnZip{ index = 0.0957 value = 0.3151 dict = 1.00 all = 0.2994 } ---------------------------- total value len = 564.774397 GB avg = 0.329 KB (by entry num) total key len = 44.702607 GB avg = 0.026 KB total ukey len = 33.071428 GB avg = 0.014 KB total ukey num = 2.334878889 Billion total entry num = 1.714380911 Billion write speed all = 47.288874856 MB/sec (with seq num) write speed all = 46.224734371 MB/sec (without seq num) 2024/11/20-21:38:30.775889 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74271: 924813 keys, 40179266 bytes, temperature: kWarm 2024/11/20-21:38:30.775950 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109910775921, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74271, "file_size": 40179266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1175942209, "table_properties": {"data_size": 34131472, "index_size": 1770624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25894764, "raw_average_key_size": 28, "raw_value_size": 108323078, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924813, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109907, "oldest_key_time": 0, "file_creation_time": 1732109907, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74271, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:30.809440 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074272.sst) /dev/shm/ToplingTemp/Topling-1024810-6jhHsy 2024/11/20-21:38:31.624239 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074272.sst): first pass time = 0.81's, 164.731'MB/sec 2024/11/20-21:38:32.793942 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074272.sst): old prealloc_size = 147639500, real_size = 35319568 2024/11/20-21:38:32.794488 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074272.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926204 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 116.91 avg-zval = 35.98 2024/11/20-21:38:32.795288 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074272.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:33.625479 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074272.sst): second pass time = 1.17's, 92.635'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.263'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3316.095'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 309.578'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33912.926'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.635'MB/sec zip pipeline throughput = 92.635'MB/sec entries = 926204 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 35.98 usrkeys = 926204 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 35.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926204 bytes = 3125952 } seq expand size = 6483428 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5132 value = 3.2493 dict = 1.00 all = 3.4152 } Zip/UnZip{ index = 0.0951 value = 0.3078 dict = 1.00 all = 0.2928 } ---------------------------- total value len = 564.889165 GB avg = 0.329 KB (by entry num) total key len = 44.728541 GB avg = 0.026 KB total ukey len = 33.089952 GB avg = 0.014 KB total ukey num = 2.335805093 Billion total entry num = 1.715307115 Billion write speed all = 47.289163737 MB/sec (with seq num) write speed all = 46.224687580 MB/sec (without seq num) 2024/11/20-21:38:33.671398 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74272: 926204 keys, 39300190 bytes, temperature: kWarm 2024/11/20-21:38:33.671461 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109913671431, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74272, "file_size": 39300190, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1053131113, "largest_seqno": 1175942268, "table_properties": {"data_size": 33325824, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933712, "raw_average_key_size": 28, "raw_value_size": 108284104, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926204, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109910, "oldest_key_time": 0, "file_creation_time": 1732109910, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74272, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:33.705913 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074273.sst) /dev/shm/ToplingTemp/Topling-1024810-2uDzZw 2024/11/20-21:38:34.511618 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074273.sst): first pass time = 0.81's, 166.591'MB/sec 2024/11/20-21:38:35.666585 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074273.sst): old prealloc_size = 147639500, real_size = 35649632 2024/11/20-21:38:35.667139 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074273.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928557 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 116.54 avg-zval = 36.24 2024/11/20-21:38:35.667892 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074273.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:36.491422 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074273.sst): second pass time = 1.15's, 93.766'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.045'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3381.116'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 304.116'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 18736.930'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 93.766'MB/sec zip pipeline throughput = 93.766'MB/sec entries = 928557 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.54 avg-zval = 36.24 usrkeys = 928557 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.20 avg-zval = 36.24 TagRS{ kinds = RS_Key0_TagN bytes = 145200 } TagArray{ size = 882270 bytes = 2977680 } seq expand size = 6175890 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5132 value = 3.2159 dict = 1.00 all = 3.3868 } Zip/UnZip{ index = 0.0951 value = 0.3110 dict = 1.00 all = 0.2953 } ---------------------------- total value len = 565.003559 GB avg = 0.329 KB (by entry num) total key len = 44.754540 GB avg = 0.026 KB total ukey len = 33.108523 GB avg = 0.014 KB total ukey num = 2.336733650 Billion total entry num = 1.716235672 Billion write speed all = 47.289541088 MB/sec (with seq num) write speed all = 46.224725418 MB/sec (without seq num) 2024/11/20-21:38:36.538123 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74273: 928557 keys, 39629440 bytes, temperature: kWarm 2024/11/20-21:38:36.538185 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109916538155, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74273, "file_size": 39629440, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1175942600, "table_properties": {"data_size": 33650832, "index_size": 1766464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25999596, "raw_average_key_size": 28, "raw_value_size": 108218210, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928557, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109913, "oldest_key_time": 0, "file_creation_time": 1732109913, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74273, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:36.571246 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074274.sst) /dev/shm/ToplingTemp/Topling-1024810-Oxsbmx 2024/11/20-21:38:37.372276 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074274.sst): first pass time = 0.80's, 167.563'MB/sec 2024/11/20-21:38:38.561164 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074274.sst): old prealloc_size = 147639500, real_size = 36345696 2024/11/20-21:38:38.561701 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074274.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923837 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0343 GB avg-val = 117.28 avg-zval = 37.18 2024/11/20-21:38:38.562228 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074274.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:39.328862 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074274.sst): second pass time = 1.19's, 91.199'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 50.885'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3443.820'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.361'MB/sec write SST data time = 0.13's, 308.511'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35093.350'MB/sec zip my value time = 1.19's, unzip length = 0.108'GB zip my value throughput = 91.199'MB/sec zip pipeline throughput = 91.199'MB/sec entries = 923837 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.28 avg-zval = 37.18 usrkeys = 923837 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.13 avg-zval = 37.18 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 507335 bytes = 1712272 } seq expand size = 3551345 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.4457 value = 3.1547 dict = 1.00 all = 3.4359 } Zip/UnZip{ index = 0.0957 value = 0.3170 dict = 1.00 all = 0.2910 } ---------------------------- total value len = 565.115460 GB avg = 0.329 KB (by entry num) total key len = 44.780408 GB avg = 0.026 KB total ukey len = 33.127000 GB avg = 0.014 KB total ukey num = 2.337657487 Billion total entry num = 1.717159509 Billion write speed all = 47.289819290 MB/sec (with seq num) write speed all = 46.224664832 MB/sec (without seq num) 2024/11/20-21:38:39.375216 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74274: 923837 keys, 39063321 bytes, temperature: kWarm 2024/11/20-21:38:39.375280 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109919375249, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74274, "file_size": 39063321, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1175942044, "table_properties": {"data_size": 34345744, "index_size": 1768832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25867436, "raw_average_key_size": 28, "raw_value_size": 108350414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923837, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109916, "oldest_key_time": 0, "file_creation_time": 1732109916, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74274, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:39.409807 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074275.sst) /dev/shm/ToplingTemp/Topling-1024810-Ni8OOx 2024/11/20-21:38:40.225964 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074275.sst): first pass time = 0.82's, 164.458'MB/sec 2024/11/20-21:38:41.396010 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074275.sst): old prealloc_size = 147639500, real_size = 35736672 2024/11/20-21:38:41.396569 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074275.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924156 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0337 GB avg-val = 117.23 avg-zval = 36.49 2024/11/20-21:38:41.397324 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074275.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:42.220956 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074275.sst): second pass time = 1.17's, 92.660'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.141'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3473.744'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 315.070'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 18329.397'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.660'MB/sec zip pipeline throughput = 92.660'MB/sec entries = 924156 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.23 avg-zval = 36.49 usrkeys = 924156 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.23 avg-zval = 36.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924156 bytes = 3119040 } seq expand size = 6469092 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.3754 value = 3.2126 dict = 1.00 all = 3.3797 } Zip/UnZip{ index = 0.0964 value = 0.3113 dict = 1.00 all = 0.2959 } ---------------------------- total value len = 565.230271 GB avg = 0.329 KB (by entry num) total key len = 44.806284 GB avg = 0.026 KB total ukey len = 33.145483 GB avg = 0.014 KB total ukey num = 2.338581643 Billion total entry num = 1.718083665 Billion write speed all = 47.290123199 MB/sec (with seq num) write speed all = 46.224634418 MB/sec (without seq num) 2024/11/20-21:38:42.267535 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74275: 924156 keys, 39712510 bytes, temperature: kWarm 2024/11/20-21:38:42.267609 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109922267577, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74275, "file_size": 39712510, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1080218076, "largest_seqno": 1175942114, "table_properties": {"data_size": 33723984, "index_size": 1781440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876368, "raw_average_key_size": 28, "raw_value_size": 108341492, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924156, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109919, "oldest_key_time": 0, "file_creation_time": 1732109919, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74275, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:42.319101 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074276.sst) /dev/shm/ToplingTemp/Topling-1024810-rzvmVv 2024/11/20-21:38:43.131075 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074276.sst): first pass time = 0.81's, 165.305'MB/sec 2024/11/20-21:38:44.241610 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074276.sst): old prealloc_size = 147639500, real_size = 35857280 2024/11/20-21:38:44.242140 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074276.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922955 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0339 GB avg-val = 117.42 avg-zval = 36.70 2024/11/20-21:38:44.242892 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074276.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:38:45.065194 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074276.sst): second pass time = 1.11's, 97.651'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.364'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3451.942'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.12's, 318.808'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 29204.411'MB/sec zip my value time = 1.11's, unzip length = 0.108'GB zip my value throughput = 97.651'MB/sec zip pipeline throughput = 97.651'MB/sec entries = 922955 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 36.70 usrkeys = 922955 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.42 avg-zval = 36.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922955 bytes = 3114992 } seq expand size = 6460685 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.5114 value = 3.1997 dict = 1.00 all = 3.3697 } Zip/UnZip{ index = 0.0951 value = 0.3125 dict = 1.00 all = 0.2968 } ---------------------------- total value len = 565.345107 GB avg = 0.329 KB (by entry num) total key len = 44.832127 GB avg = 0.026 KB total ukey len = 33.163942 GB avg = 0.014 KB total ukey num = 2.339504598 Billion total entry num = 1.719006620 Billion write speed all = 47.290601707 MB/sec (with seq num) write speed all = 46.224775545 MB/sec (without seq num) 2024/11/20-21:38:45.112599 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74276: 922955 keys, 39830448 bytes, temperature: kWarm 2024/11/20-21:38:45.112660 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109925112630, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74276, "file_size": 39830448, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1080218137, "largest_seqno": 1175942154, "table_properties": {"data_size": 33870256, "index_size": 1756096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842740, "raw_average_key_size": 28, "raw_value_size": 108375018, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922955, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109922, "oldest_key_time": 0, "file_creation_time": 1732109922, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74276, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:45.147015 140008937477696 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074277.sst) /dev/shm/ToplingTemp/Topling-1024810-EVD5Bv 2024/11/20-21:38:45.379867 140008937477696 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074277.sst): first pass time = 0.23's, 255.823'MB/sec 2024/11/20-21:38:45.916134 140008937477696 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074277.sst): old prealloc_size = 147639500, real_size = 16496928 2024/11/20-21:38:45.916396 140008937477696 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074277.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 410340 prefix = 4 raw-key = 0.0066 GB zip-key = 0.0008 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0481 GB zip-val = 0.0156 GB avg-val = 117.15 avg-zval = 38.05 2024/11/20-21:38:45.916794 140008937477696 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074277.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0010 GB, waited 0.000 sec, Key+Value = 0.060 GB 2024/11/20-21:38:46.281566 140008937477696 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00/074277.sst): second pass time = 0.54's, 89.763'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 50.150'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3240.997'MB/sec (index lex order gen) rebuild zvType time = 0.31's, 0.333'MB/sec write SST data time = 0.06's, 321.874'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.489'MB, throughput = 16086.388'MB/sec zip my value time = 0.54's, unzip length = 0.048'GB zip my value throughput = 89.763'MB/sec zip pipeline throughput = 89.763'MB/sec entries = 410340 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 38.05 usrkeys = 410340 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 38.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 410340 bytes = 1436208 } seq expand size = 2872380 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0082 GB value = 0.0481 GB dict = 0.49 MB all = 0.0596 GB } __ZipSize{ index = 0.0008 GB value = 0.0156 GB dict = 0.49 MB all = 0.0183 GB } UnZip/Zip{ index = 10.4987 value = 3.0791 dict = 1.00 all = 3.2504 } Zip/UnZip{ index = 0.0952 value = 0.3248 dict = 1.00 all = 0.3077 } ---------------------------- total value len = 565.396051 GB avg = 0.329 KB (by entry num) total key len = 44.843616 GB avg = 0.026 KB total ukey len = 33.172149 GB avg = 0.014 KB total ukey num = 2.339914938 Billion total entry num = 1.719416960 Billion write speed all = 47.290982351 MB/sec (with seq num) write speed all = 46.225002260 MB/sec (without seq num) 2024/11/20-21:38:46.307130 140008937477696 [compaction_job.cc:1948] [default] [JOB 14817] Generated table #74277: 410340 keys, 18324707 bytes, temperature: kWarm 2024/11/20-21:38:46.307182 140008937477696 EVENT_LOG_v1 {"time_micros": 1732109926307154, "cf_name": "default", "job": 14817, "event": "table_file_creation", "file_number": 74277, "file_size": 18324707, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 951052351, "largest_seqno": 1175942168, "table_properties": {"data_size": 15612432, "index_size": 781696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 11489520, "raw_average_key_size": 28, "raw_value_size": 48072400, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 410340, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732109925, "oldest_key_time": 0, "file_creation_time": 1732109925, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 74277, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:38:46.321447 140008937477696 [compaction_job.cc:703] job-14817: subcompact[0], size: 0.813548 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14817/att-00 [074257,074258,074259,074260,074261,074262,074263,074264,074265,074266,074267,074268,074269,074270,074271,074272,074273,074274,074275,074276,074277] 2024/11/20-21:38:46.464741 140008937477696 (Original Log Time 2024/11/20-21:37:42.026727) EVENT_LOG_v1 {"time_micros": 1732109862026703, "job": 14817, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [74253], "files_L3": [74219, 74220, 74221, 74222, 74223, 74224, 74225, 74226, 74227, 74228, 74229, 74230, 74231, 74232, 74233, 74133, 74134, 74135, 74191, 74192, 74193], "score": 0.934188, "input_data_size": 858395960, "oldest_snapshot_seqno": 1176745065} 2024/11/20-21:38:46.464744 140008937477696 (Original Log Time 2024/11/20-21:38:46.463179) [compaction_job.cc:2022] [default] [JOB 14817] Compacted 1@2 + 21@3 files to L3 => 813547599 bytes 2024/11/20-21:38:46.464746 140008937477696 (Original Log Time 2024/11/20-21:38:46.464671) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 13.4 rd, 12.7 wr, level 3, files in(1, 21) out(21 +0 blob) MB in(54.4, 764.3 +0.0 blob) out(775.9 +0.0 blob), read-write-amplify(29.3) write-amplify(14.3) OK, records in: 19090941, records dropped: 190579 output_compression: Snappy 2024/11/20-21:38:46.464750 140008937477696 (Original Log Time 2024/11/20-21:38:46.464720) EVENT_LOG_v1 {"time_micros": 1732109926464692, "cf": "default", "job": 14817, "event": "compaction_finished", "compaction_time_micros": 64294726, "compaction_time_cpu_micros": 29071772, "output_level": 3, "num_output_files": 21, "total_output_size": 813547599, "num_input_records": 19090941, "num_output_records": 18900362, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 574783701, "file_range_sync_nanos": 0, "file_fsync_nanos": 18409, "file_prepare_write_nanos": 71713, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/20-21:38:46.592037 140008937477696 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732109861,"executesMs":64547,"compactionJobId":14817,"attempt":0,"compactionInputRawBytes":2770980054,"compactionInputZipBytes":858395960,"compactionOutputRawBytes":0,"compactionOutputZipBytes":813547599,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-4756a9df934df29f631bad389f9743f8-860dd760ba853171-00"}}, response = 2024/11/20-21:38:46.592108 140008937477696 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-14817/att-00: olev 3, work 64.548 s, result 7.725 ms, install 1.586 ms, input{raw 2.581 GiB zip 818.630 MiB}