2024/11/21-03:37:26.340192 140008660518464 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00: fork to child time = 0.030411 sec 2024/11/21-03:37:26.340882 140008660518464 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00 2024/11/21-03:37:26.348414 140008660518464 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00 2024/11/21-03:37:26.359433 140008660518464 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/21-03:37:26.359809 140008660518464 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/21-03:37:26.412207 140008660518464 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-21T00.56.43/.rocksdb/job-04291/att-00/MANIFEST-090329 2024/11/21-03:37:26.427078 140008660518464 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-21T00.56.43/.rocksdb/job-04291/att-00/MANIFEST-090329 succeeded,manifest_file_number is 90329, next_file_number is 99374, last_sequence is 1496250564, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 99219 2024/11/21-03:37:26.427088 140008660518464 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/21-03:37:26.427204 140008660518464 [version_set.cc:5724] Creating manifest 99374 2024/11/21-03:37:26.467608 140008660518464 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/21-03:37:26.491179 140008660518464 [compaction_job.cc:2437] [default] [JOB 4291] Compacting 2@2 + 18@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/21-03:37:26.491202 140008660518464 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 3225 Base level 2, inputs: [99368(27MB) 99366(4062KB)], [99300(36MB) 99301(36MB) 99302(36MB) 99303(37MB) 99304(38MB) 99305(35MB) 99306(39MB) 99307(38MB) 99308(38MB) 99309(38MB) 99310(38MB) 99311(38MB) 99312(36MB) 99313(37MB) 99314(36MB) 99315(38MB) 99316(36MB) 99324(38MB)] 2024/11/21-03:37:26.500000 140008660518464 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/21-03:37:26.500026 140008660518464 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/21-03:37:26.500052 140008660518464 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/21-03:37:26.500061 140008660518464 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/21-03:37:26.500072 140008660518464 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/21-03:37:27.078607 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099375.sst) /dev/shm/ToplingTemp/Topling-1304093-oRPEIA 2024/11/21-03:37:27.748185 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099375.sst): first pass time = 0.67's, 200.465'MB/sec 2024/11/21-03:37:29.030784 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099375.sst): old prealloc_size = 147639500, real_size = 33843968 2024/11/21-03:37:29.031442 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099375.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923323 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.36 avg-zval = 34.44 2024/11/21-03:37:29.033035 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099375.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:37:30.004644 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099375.sst): second pass time = 1.28's, 84.552'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 39.193'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2870.562'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.274'MB/sec write SST data time = 0.13's, 290.510'MB/sec dict compress time = 2316948516.44's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 18642.837'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.552'MB/sec zip pipeline throughput = 84.552'MB/sec entries = 923323 avg-key = 28.00 avg-zkey = 1.97 avg-val = 117.36 avg-zval = 34.44 usrkeys = 923323 avg-key = 20.00 avg-zkey = 1.97 avg-val = 124.36 avg-zval = 34.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923323 bytes = 3462480 } seq expand size = 6463261 multi value expand size = 0 cnt WriteAppend = 82 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.1770 value = 3.4079 dict = 1.00 all = 3.5165 } Zip/UnZip{ index = 0.0983 value = 0.2934 dict = 1.00 all = 0.2844 } ---------------------------- total value len = 563.180560 GB avg = 0.331 KB (by entry num) total key len = 44.340259 GB avg = 0.026 KB total ukey len = 32.812608 GB avg = 0.014 KB total ukey num = 2.321937899 Billion total entry num = 1.701439921 Billion write speed all = 17.646302092 MB/sec (with seq num) write speed all = 17.250936246 MB/sec (without seq num) 2024/11/21-03:37:30.053053 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99375: 923323 keys, 38168040 bytes, temperature: kWarm 2024/11/21-03:37:30.053151 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131450053106, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99375, "file_size": 38168040, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 876291829, "largest_seqno": 1496200305, "table_properties": {"data_size": 31798448, "index_size": 1814528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25853044, "raw_average_key_size": 28, "raw_value_size": 108364790, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923323, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131447, "oldest_key_time": 0, "file_creation_time": 1732131447, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99375, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:37:30.092106 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099376.sst) /dev/shm/ToplingTemp/Topling-1304093-T4p3My 2024/11/21-03:37:31.086319 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099376.sst): first pass time = 0.99's, 135.005'MB/sec 2024/11/21-03:37:32.272572 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099376.sst): old prealloc_size = 147639500, real_size = 33887648 2024/11/21-03:37:32.273335 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099376.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921274 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.69 avg-zval = 34.63 2024/11/21-03:37:32.274178 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099376.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:37:33.186198 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099376.sst): second pass time = 1.19's, 91.475'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.435'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2441.079'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.303'MB/sec write SST data time = 0.15's, 248.551'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 19637.438'MB/sec zip my value time = 1.19's, unzip length = 0.108'GB zip my value throughput = 91.475'MB/sec zip pipeline throughput = 91.475'MB/sec entries = 921274 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.69 avg-zval = 34.63 usrkeys = 921274 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.69 avg-zval = 34.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921274 bytes = 3109312 } seq expand size = 6448918 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5138 value = 3.3983 dict = 1.00 all = 3.5455 } Zip/UnZip{ index = 0.0951 value = 0.2943 dict = 1.00 all = 0.2821 } ---------------------------- total value len = 563.295431 GB avg = 0.331 KB (by entry num) total key len = 44.366055 GB avg = 0.026 KB total ukey len = 32.831034 GB avg = 0.014 KB total ukey num = 2.322859173 Billion total entry num = 1.702361195 Billion write speed all = 17.648756984 MB/sec (with seq num) write speed all = 17.253213614 MB/sec (without seq num) 2024/11/21-03:37:33.234092 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99376: 921274 keys, 37856188 bytes, temperature: kWarm 2024/11/21-03:37:33.234173 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131453234142, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99376, "file_size": 37856188, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1375096932, "largest_seqno": 1496200317, "table_properties": {"data_size": 31904656, "index_size": 1752512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25795672, "raw_average_key_size": 28, "raw_value_size": 108422084, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921274, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131450, "oldest_key_time": 0, "file_creation_time": 1732131450, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99376, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:37:33.269594 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099377.sst) /dev/shm/ToplingTemp/Topling-1304093-VgAAhC 2024/11/21-03:37:34.274130 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099377.sst): first pass time = 1.00's, 133.617'MB/sec 2024/11/21-03:37:35.469162 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099377.sst): old prealloc_size = 147639500, real_size = 34869120 2024/11/21-03:37:35.469822 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099377.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923548 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0329 GB avg-val = 117.33 avg-zval = 35.60 2024/11/21-03:37:35.470659 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099377.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:37:36.349211 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099377.sst): second pass time = 1.19's, 90.741'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 44.138'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2793.946'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.311'MB/sec write SST data time = 0.14's, 284.145'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 39722.514'MB/sec zip my value time = 1.19's, unzip length = 0.108'GB zip my value throughput = 90.741'MB/sec zip pipeline throughput = 90.741'MB/sec entries = 923548 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 35.60 usrkeys = 923548 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 35.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923548 bytes = 3116992 } seq expand size = 6464836 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5140 value = 3.2954 dict = 1.00 all = 3.4553 } Zip/UnZip{ index = 0.0951 value = 0.3034 dict = 1.00 all = 0.2894 } ---------------------------- total value len = 563.410255 GB avg = 0.331 KB (by entry num) total key len = 44.391914 GB avg = 0.026 KB total ukey len = 32.849505 GB avg = 0.014 KB total ukey num = 2.323782721 Billion total entry num = 1.703284743 Billion write speed all = 17.651221388 MB/sec (with seq num) write speed all = 17.255499785 MB/sec (without seq num) 2024/11/21-03:37:36.401134 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99377: 923548 keys, 38843900 bytes, temperature: kWarm 2024/11/21-03:37:36.401226 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131456401181, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99377, "file_size": 38843900, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1428168205, "largest_seqno": 1496201120, "table_properties": {"data_size": 32881264, "index_size": 1756800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859344, "raw_average_key_size": 28, "raw_value_size": 108358396, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923548, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131453, "oldest_key_time": 0, "file_creation_time": 1732131453, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99377, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:37:36.460005 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099378.sst) /dev/shm/ToplingTemp/Topling-1304093-YirH8y 2024/11/21-03:37:37.744776 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099378.sst): first pass time = 1.28's, 104.481'MB/sec 2024/11/21-03:37:39.025323 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099378.sst): old prealloc_size = 147639500, real_size = 35149616 2024/11/21-03:37:39.025940 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099378.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927314 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 116.74 avg-zval = 35.75 2024/11/21-03:37:39.026799 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099378.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:37:39.872330 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099378.sst): second pass time = 1.28's, 84.590'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.203'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2995.107'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 291.989'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 23742.787'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.590'MB/sec zip pipeline throughput = 85.793'MB/sec entries = 927314 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 35.75 usrkeys = 927314 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.74 avg-zval = 35.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927314 bytes = 3129696 } seq expand size = 6491198 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5140 value = 3.2652 dict = 1.00 all = 3.4297 } Zip/UnZip{ index = 0.0951 value = 0.3063 dict = 1.00 all = 0.2916 } ---------------------------- total value len = 563.524999 GB avg = 0.331 KB (by entry num) total key len = 44.417879 GB avg = 0.026 KB total ukey len = 32.868051 GB avg = 0.014 KB total ukey num = 2.324710035 Billion total entry num = 1.704212057 Billion write speed all = 17.653501511 MB/sec (with seq num) write speed all = 17.257604973 MB/sec (without seq num) 2024/11/21-03:37:39.921793 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99378: 927314 keys, 39133886 bytes, temperature: kWarm 2024/11/21-03:37:39.921858 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131459921827, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99378, "file_size": 39133886, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1428168229, "largest_seqno": 1496200921, "table_properties": {"data_size": 33153632, "index_size": 1763968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25964792, "raw_average_key_size": 28, "raw_value_size": 108252972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927314, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131456, "oldest_key_time": 0, "file_creation_time": 1732131456, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99378, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:37:39.961275 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099379.sst) /dev/shm/ToplingTemp/Topling-1304093-0x8u3B 2024/11/21-03:37:41.075462 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099379.sst): first pass time = 1.11's, 120.467'MB/sec 2024/11/21-03:37:42.457240 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099379.sst): old prealloc_size = 147639500, real_size = 36590976 2024/11/21-03:37:42.457825 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099379.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928409 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.96 raw-val = 0.1082 GB zip-val = 0.0345 GB avg-val = 116.57 avg-zval = 37.20 2024/11/21-03:37:42.458621 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099379.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:37:43.411501 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099379.sst): second pass time = 1.38's, 78.370'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 42.729'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3242.326'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.339'MB/sec write SST data time = 0.27's, 151.785'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 28335.477'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.370'MB/sec zip pipeline throughput = 125.232'MB/sec entries = 928409 avg-key = 28.00 avg-zkey = 1.96 avg-val = 116.57 avg-zval = 37.20 usrkeys = 928409 avg-key = 20.00 avg-zkey = 1.96 avg-val = 122.96 avg-zval = 37.20 TagRS{ kinds = RS_Key0_TagN bytes = 145160 } TagArray{ size = 847917 bytes = 3179696 } seq expand size = 5935419 multi value expand size = 0 cnt WriteAppend = 27 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0408 GB } UnZip/Zip{ index = 10.1785 value = 3.1338 dict = 1.00 all = 3.2919 } Zip/UnZip{ index = 0.0982 value = 0.3191 dict = 1.00 all = 0.3038 } ---------------------------- total value len = 563.639157 GB avg = 0.331 KB (by entry num) total key len = 44.443875 GB avg = 0.026 KB total ukey len = 32.886619 GB avg = 0.014 KB total ukey num = 2.325638444 Billion total entry num = 1.705140466 Billion write speed all = 17.655756812 MB/sec (with seq num) write speed all = 17.259685304 MB/sec (without seq num) 2024/11/21-03:37:43.464212 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99379: 928409 keys, 40772598 bytes, temperature: kWarm 2024/11/21-03:37:43.464293 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131463464248, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99379, "file_size": 40772598, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1496201294, "table_properties": {"data_size": 34534448, "index_size": 1824256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995452, "raw_average_key_size": 28, "raw_value_size": 108222402, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928409, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131459, "oldest_key_time": 0, "file_creation_time": 1732131459, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99379, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:37:43.539091 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099380.sst) /dev/shm/ToplingTemp/Topling-1304093-RGjvfC 2024/11/21-03:37:44.425088 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099380.sst): first pass time = 0.89's, 151.498'MB/sec 2024/11/21-03:37:45.845637 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099380.sst): old prealloc_size = 147639500, real_size = 36514400 2024/11/21-03:37:45.846221 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099380.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926782 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.82 avg-zval = 37.18 2024/11/21-03:37:45.846394 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099380.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:37:46.628116 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099380.sst): second pass time = 1.42's, 76.257'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 48.265'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3233.762'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.353'MB/sec write SST data time = 0.13's, 299.635'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18150.519'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.257'MB/sec zip pipeline throughput = 76.257'MB/sec entries = 926782 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.82 avg-zval = 37.18 usrkeys = 926782 avg-key = 20.00 avg-zkey = 1.97 avg-val = 117.38 avg-zval = 37.18 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 73430 bytes = 247840 } seq expand size = 514010 multi value expand size = 0 cnt WriteAppend = 65 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.1742 value = 3.1418 dict = 1.00 all = 3.5540 } Zip/UnZip{ index = 0.0983 value = 0.3183 dict = 1.00 all = 0.2814 } ---------------------------- total value len = 563.747938 GB avg = 0.330 KB (by entry num) total key len = 44.469825 GB avg = 0.026 KB total ukey len = 32.905155 GB avg = 0.014 KB total ukey num = 2.326565226 Billion total entry num = 1.706067248 Billion write speed all = 17.658019579 MB/sec (with seq num) write speed all = 17.261769805 MB/sec (without seq num) 2024/11/21-03:37:46.674923 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99380: 926782 keys, 37765281 bytes, temperature: kWarm 2024/11/21-03:37:46.674980 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131466674951, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99380, "file_size": 37765281, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 978567417, "table_properties": {"data_size": 34460688, "index_size": 1821824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949896, "raw_average_key_size": 28, "raw_value_size": 108267952, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926782, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131463, "oldest_key_time": 0, "file_creation_time": 1732131463, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99380, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:37:46.711221 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099381.sst) /dev/shm/ToplingTemp/Topling-1304093-rlHIPz 2024/11/21-03:37:47.685267 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099381.sst): first pass time = 0.97's, 137.798'MB/sec 2024/11/21-03:37:48.971394 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099381.sst): old prealloc_size = 147639500, real_size = 37485072 2024/11/21-03:37:48.972151 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099381.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927613 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1082 GB zip-val = 0.0355 GB avg-val = 116.69 avg-zval = 38.22 2024/11/21-03:37:48.972815 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099381.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:37:49.760957 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099381.sst): second pass time = 1.29's, 84.225'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 51.435'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2513.883'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.368'MB/sec write SST data time = 0.16's, 257.283'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35347.947'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.225'MB/sec zip pipeline throughput = 84.225'MB/sec entries = 927613 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.69 avg-zval = 38.22 usrkeys = 927613 avg-key = 20.00 avg-zkey = 1.94 avg-val = 121.11 avg-zval = 38.22 TagRS{ kinds = RS_Key0_TagN bytes = 145040 } TagArray{ size = 585374 bytes = 2122000 } seq expand size = 4097618 multi value expand size = 0 cnt WriteAppend = 25 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0355 GB dict = 1.08 MB all = 0.0406 GB } UnZip/Zip{ index = 10.3090 value = 3.0532 dict = 1.00 all = 3.3050 } Zip/UnZip{ index = 0.0970 value = 0.3275 dict = 1.00 all = 0.3026 } ---------------------------- total value len = 563.860281 GB avg = 0.330 KB (by entry num) total key len = 44.495798 GB avg = 0.026 KB total ukey len = 32.923707 GB avg = 0.014 KB total ukey num = 2.327492839 Billion total entry num = 1.706994861 Billion write speed all = 17.660428925 MB/sec (with seq num) write speed all = 17.263999762 MB/sec (without seq num) 2024/11/21-03:37:49.809821 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99381: 927613 keys, 40610282 bytes, temperature: kWarm 2024/11/21-03:37:49.809894 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131469809861, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99381, "file_size": 40610282, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1323177458, "table_properties": {"data_size": 35453376, "index_size": 1799616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25973164, "raw_average_key_size": 28, "raw_value_size": 108244598, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927613, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131466, "oldest_key_time": 0, "file_creation_time": 1732131466, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99381, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:37:49.843956 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099382.sst) /dev/shm/ToplingTemp/Topling-1304093-fziVYz 2024/11/21-03:37:50.870882 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099382.sst): first pass time = 1.03's, 130.705'MB/sec 2024/11/21-03:37:52.058810 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099382.sst): old prealloc_size = 147639500, real_size = 37217616 2024/11/21-03:37:52.059453 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099382.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925251 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0352 GB avg-val = 117.06 avg-zval = 38.07 2024/11/21-03:37:52.060310 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099382.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:37:52.940399 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099382.sst): second pass time = 1.19's, 91.242'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 46.995'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2911.016'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 319.262'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35158.614'MB/sec zip my value time = 1.19's, unzip length = 0.108'GB zip my value throughput = 91.242'MB/sec zip pipeline throughput = 91.242'MB/sec entries = 925251 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.06 avg-zval = 38.07 usrkeys = 925251 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.06 avg-zval = 38.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925251 bytes = 3354048 } seq expand size = 6476757 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0414 GB } UnZip/Zip{ index = 10.5150 value = 3.0747 dict = 1.00 all = 3.2396 } Zip/UnZip{ index = 0.0951 value = 0.3252 dict = 1.00 all = 0.3087 } ---------------------------- total value len = 563.975068 GB avg = 0.330 KB (by entry num) total key len = 44.521705 GB avg = 0.026 KB total ukey len = 32.942212 GB avg = 0.014 KB total ukey num = 2.328418090 Billion total entry num = 1.707920112 Billion write speed all = 17.662882996 MB/sec (with seq num) write speed all = 17.266275560 MB/sec (without seq num) 2024/11/21-03:37:52.992751 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99382: 925251 keys, 41430504 bytes, temperature: kWarm 2024/11/21-03:37:52.992822 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131472992791, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99382, "file_size": 41430504, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1052137445, "largest_seqno": 1331715475, "table_properties": {"data_size": 35226240, "index_size": 1759872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25907028, "raw_average_key_size": 28, "raw_value_size": 108310758, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925251, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131469, "oldest_key_time": 0, "file_creation_time": 1732131469, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99382, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:37:53.031394 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099383.sst) /dev/shm/ToplingTemp/Topling-1304093-JobTWB 2024/11/21-03:37:54.021778 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099383.sst): first pass time = 0.99's, 135.528'MB/sec 2024/11/21-03:37:55.217851 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099383.sst): old prealloc_size = 147639500, real_size = 35612272 2024/11/21-03:37:55.218479 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099383.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928178 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0336 GB avg-val = 116.60 avg-zval = 36.22 2024/11/21-03:37:55.219503 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099383.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:37:56.113416 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099383.sst): second pass time = 1.20's, 90.554'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 44.346'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3005.391'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.302'MB/sec write SST data time = 0.13's, 312.647'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27228.907'MB/sec zip my value time = 1.20's, unzip length = 0.108'GB zip my value throughput = 90.554'MB/sec zip pipeline throughput = 90.554'MB/sec entries = 928178 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.60 avg-zval = 36.22 usrkeys = 928178 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.60 avg-zval = 36.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928178 bytes = 3248640 } seq expand size = 6497246 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5115 value = 3.2198 dict = 1.00 all = 3.3796 } Zip/UnZip{ index = 0.0951 value = 0.3106 dict = 1.00 all = 0.2959 } ---------------------------- total value len = 564.089794 GB avg = 0.330 KB (by entry num) total key len = 44.547694 GB avg = 0.026 KB total ukey len = 32.960776 GB avg = 0.014 KB total ukey num = 2.329346268 Billion total entry num = 1.708848290 Billion write speed all = 17.665340507 MB/sec (with seq num) write speed all = 17.268554079 MB/sec (without seq num) 2024/11/21-03:37:56.161319 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99383: 928178 keys, 39714258 bytes, temperature: kWarm 2024/11/21-03:37:56.161393 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131476161361, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99383, "file_size": 39714258, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187855354, "largest_seqno": 1331717282, "table_properties": {"data_size": 33614048, "index_size": 1766016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25988984, "raw_average_key_size": 28, "raw_value_size": 108228860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131473, "oldest_key_time": 0, "file_creation_time": 1732131473, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99383, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:37:56.198246 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099384.sst) /dev/shm/ToplingTemp/Topling-1304093-EzZNez 2024/11/21-03:37:57.119726 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099384.sst): first pass time = 0.92's, 145.662'MB/sec 2024/11/21-03:37:58.247994 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099384.sst): old prealloc_size = 147639500, real_size = 35554624 2024/11/21-03:37:58.248645 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099384.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928475 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.56 avg-zval = 36.13 2024/11/21-03:37:58.249496 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099384.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:37:59.077463 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099384.sst): second pass time = 1.13's, 95.983'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.816'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3041.742'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 311.624'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23810.137'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 95.983'MB/sec zip pipeline throughput = 95.983'MB/sec entries = 928475 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.56 avg-zval = 36.13 usrkeys = 928475 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.56 avg-zval = 36.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928475 bytes = 3249680 } seq expand size = 6499325 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.4453 value = 3.2262 dict = 1.00 all = 3.3841 } Zip/UnZip{ index = 0.0957 value = 0.3100 dict = 1.00 all = 0.2955 } ---------------------------- total value len = 564.204514 GB avg = 0.330 KB (by entry num) total key len = 44.573691 GB avg = 0.026 KB total ukey len = 32.979345 GB avg = 0.014 KB total ukey num = 2.330274743 Billion total entry num = 1.709776765 Billion write speed all = 17.667904777 MB/sec (with seq num) write speed all = 17.270936912 MB/sec (without seq num) 2024/11/21-03:37:59.125568 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99384: 928475 keys, 39661280 bytes, temperature: kWarm 2024/11/21-03:37:59.125643 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131479125609, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99384, "file_size": 39661280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187855426, "largest_seqno": 1331716806, "table_properties": {"data_size": 33544560, "index_size": 1777792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997300, "raw_average_key_size": 28, "raw_value_size": 108220522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928475, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131476, "oldest_key_time": 0, "file_creation_time": 1732131476, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99384, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:37:59.162539 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099385.sst) /dev/shm/ToplingTemp/Topling-1304093-g5sdtB 2024/11/21-03:37:59.976291 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099385.sst): first pass time = 0.81's, 164.947'MB/sec 2024/11/21-03:38:01.104944 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099385.sst): old prealloc_size = 147639500, real_size = 36239392 2024/11/21-03:38:01.105568 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099385.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927448 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0342 GB avg-val = 116.72 avg-zval = 36.91 2024/11/21-03:38:01.106364 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099385.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:38:01.922671 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099385.sst): second pass time = 1.13's, 95.989'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.332'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3034.196'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 321.632'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33379.391'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 95.989'MB/sec zip pipeline throughput = 95.989'MB/sec entries = 927448 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.72 avg-zval = 36.91 usrkeys = 927448 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.72 avg-zval = 36.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927448 bytes = 3246080 } seq expand size = 6492136 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0403 GB } UnZip/Zip{ index = 10.4446 value = 3.1623 dict = 1.00 all = 3.3272 } Zip/UnZip{ index = 0.0957 value = 0.3162 dict = 1.00 all = 0.3006 } ---------------------------- total value len = 564.319256 GB avg = 0.330 KB (by entry num) total key len = 44.599660 GB avg = 0.026 KB total ukey len = 32.997894 GB avg = 0.014 KB total ukey num = 2.331202191 Billion total entry num = 1.710704213 Billion write speed all = 17.670529330 MB/sec (with seq num) write speed all = 17.273378928 MB/sec (without seq num) 2024/11/21-03:38:01.970325 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99385: 927448 keys, 40339796 bytes, temperature: kWarm 2024/11/21-03:38:01.970396 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131481970363, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99385, "file_size": 40339796, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187851486, "largest_seqno": 1331717093, "table_properties": {"data_size": 34231440, "index_size": 1775936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25968544, "raw_average_key_size": 28, "raw_value_size": 108249272, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927448, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131479, "oldest_key_time": 0, "file_creation_time": 1732131479, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99385, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:38:02.006107 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099386.sst) /dev/shm/ToplingTemp/Topling-1304093-Usz37z 2024/11/21-03:38:03.041983 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099386.sst): first pass time = 1.04's, 129.575'MB/sec 2024/11/21-03:38:04.232525 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099386.sst): old prealloc_size = 147639500, real_size = 36664720 2024/11/21-03:38:04.233116 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099386.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928966 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0347 GB avg-val = 116.48 avg-zval = 37.33 2024/11/21-03:38:04.233979 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099386.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:38:05.052691 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099386.sst): second pass time = 1.19's, 90.961'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.961'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3181.165'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 317.089'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 18820.106'MB/sec zip my value time = 1.19's, unzip length = 0.108'GB zip my value throughput = 90.961'MB/sec zip pipeline throughput = 90.961'MB/sec entries = 928966 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.48 avg-zval = 37.33 usrkeys = 928966 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.36 avg-zval = 37.33 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 912935 bytes = 3309408 } seq expand size = 6390545 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0410 GB } UnZip/Zip{ index = 10.5838 value = 3.1204 dict = 1.00 all = 3.2755 } Zip/UnZip{ index = 0.0945 value = 0.3205 dict = 1.00 all = 0.3053 } ---------------------------- total value len = 564.433853 GB avg = 0.330 KB (by entry num) total key len = 44.625671 GB avg = 0.026 KB total ukey len = 33.016473 GB avg = 0.014 KB total ukey num = 2.332131157 Billion total entry num = 1.711633179 Billion write speed all = 17.673004447 MB/sec (with seq num) write speed all = 17.275674471 MB/sec (without seq num) 2024/11/21-03:38:05.103614 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99386: 928966 keys, 40976166 bytes, temperature: kWarm 2024/11/21-03:38:05.103688 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131485103655, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99386, "file_size": 40976166, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1375096600, "table_properties": {"data_size": 34676864, "index_size": 1755456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26011048, "raw_average_key_size": 28, "raw_value_size": 108206784, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928966, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131481, "oldest_key_time": 0, "file_creation_time": 1732131481, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99386, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:38:05.138894 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099387.sst) /dev/shm/ToplingTemp/Topling-1304093-CH19kz 2024/11/21-03:38:05.998819 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099387.sst): first pass time = 0.86's, 156.088'MB/sec 2024/11/21-03:38:07.124968 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099387.sst): old prealloc_size = 147639500, real_size = 33933536 2024/11/21-03:38:07.125604 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099387.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927642 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0319 GB avg-val = 116.69 avg-zval = 34.43 2024/11/21-03:38:07.126383 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099387.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:38:07.970788 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099387.sst): second pass time = 1.13's, 96.191'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.831'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2963.175'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 305.353'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18892.775'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 96.191'MB/sec zip pipeline throughput = 96.191'MB/sec entries = 927642 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.69 avg-zval = 34.43 usrkeys = 927642 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.69 avg-zval = 34.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927642 bytes = 3130800 } seq expand size = 6493494 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5142 value = 3.3893 dict = 1.00 all = 3.5396 } Zip/UnZip{ index = 0.0951 value = 0.2950 dict = 1.00 all = 0.2825 } ---------------------------- total value len = 564.548590 GB avg = 0.330 KB (by entry num) total key len = 44.651645 GB avg = 0.026 KB total ukey len = 33.035026 GB avg = 0.014 KB total ukey num = 2.333058799 Billion total entry num = 1.712560821 Billion write speed all = 17.675590788 MB/sec (with seq num) write speed all = 17.278079132 MB/sec (without seq num) 2024/11/21-03:38:08.019624 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99387: 927642 keys, 37918494 bytes, temperature: kWarm 2024/11/21-03:38:08.019697 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131488019664, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99387, "file_size": 37918494, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1270579059, "largest_seqno": 1375096758, "table_properties": {"data_size": 31936912, "index_size": 1764544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25973976, "raw_average_key_size": 28, "raw_value_size": 108243872, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927642, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131485, "oldest_key_time": 0, "file_creation_time": 1732131485, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99387, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:38:08.053638 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099388.sst) /dev/shm/ToplingTemp/Topling-1304093-FN3evz 2024/11/21-03:38:09.066831 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099388.sst): first pass time = 1.01's, 132.476'MB/sec 2024/11/21-03:38:10.202003 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099388.sst): old prealloc_size = 147639500, real_size = 34413728 2024/11/21-03:38:10.202650 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099388.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931277 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0324 GB avg-val = 116.12 avg-zval = 34.80 2024/11/21-03:38:10.203454 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099388.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:38:11.038057 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099388.sst): second pass time = 1.13's, 95.341'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.219'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3062.236'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.12's, 310.449'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27520.086'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 95.341'MB/sec zip pipeline throughput = 95.341'MB/sec entries = 931277 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 34.80 usrkeys = 931277 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 34.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931277 bytes = 3375888 } seq expand size = 6518939 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5127 value = 3.3368 dict = 1.00 all = 3.4735 } Zip/UnZip{ index = 0.0951 value = 0.2997 dict = 1.00 all = 0.2879 } ---------------------------- total value len = 564.663251 GB avg = 0.330 KB (by entry num) total key len = 44.677720 GB avg = 0.026 KB total ukey len = 33.053652 GB avg = 0.014 KB total ukey num = 2.333990076 Billion total entry num = 1.713492098 Billion write speed all = 17.678100924 MB/sec (with seq num) write speed all = 17.280408497 MB/sec (without seq num) 2024/11/21-03:38:11.087939 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99388: 931277 keys, 38640932 bytes, temperature: kWarm 2024/11/21-03:38:11.088008 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131491087977, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99388, "file_size": 38640932, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1053131138, "largest_seqno": 1375096388, "table_properties": {"data_size": 32409040, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26075756, "raw_average_key_size": 28, "raw_value_size": 108142014, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931277, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131488, "oldest_key_time": 0, "file_creation_time": 1732131488, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99388, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:38:11.122447 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099389.sst) /dev/shm/ToplingTemp/Topling-1304093-sOfDcD 2024/11/21-03:38:12.102309 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099389.sst): first pass time = 0.98's, 136.981'MB/sec 2024/11/21-03:38:13.320835 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099389.sst): old prealloc_size = 147639500, real_size = 36697184 2024/11/21-03:38:13.321397 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099389.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926320 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0347 GB avg-val = 116.89 avg-zval = 37.46 2024/11/21-03:38:13.321892 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099389.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:38:14.076919 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099389.sst): second pass time = 1.22's, 88.921'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 52.087'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3286.983'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.367'MB/sec write SST data time = 0.12's, 315.331'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32134.158'MB/sec zip my value time = 1.22's, unzip length = 0.108'GB zip my value throughput = 88.921'MB/sec zip pipeline throughput = 88.921'MB/sec entries = 926320 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.89 avg-zval = 37.46 usrkeys = 926320 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.40 avg-zval = 37.46 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 463531 bytes = 1680320 } seq expand size = 3244717 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.5145 value = 3.1202 dict = 1.00 all = 3.4082 } Zip/UnZip{ index = 0.0951 value = 0.3205 dict = 1.00 all = 0.2934 } ---------------------------- total value len = 564.774777 GB avg = 0.329 KB (by entry num) total key len = 44.703657 GB avg = 0.026 KB total ukey len = 33.072178 GB avg = 0.014 KB total ukey num = 2.334916396 Billion total entry num = 1.714418418 Billion write speed all = 17.680530189 MB/sec (with seq num) write speed all = 17.282657846 MB/sec (without seq num) 2024/11/21-03:38:14.126300 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99389: 926320 keys, 39380621 bytes, temperature: kWarm 2024/11/21-03:38:14.126372 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131494126338, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99389, "file_size": 39380621, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1375096137, "table_properties": {"data_size": 34703440, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25936960, "raw_average_key_size": 28, "raw_value_size": 108280788, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926320, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131491, "oldest_key_time": 0, "file_creation_time": 1732131491, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99389, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:38:14.160243 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099390.sst) /dev/shm/ToplingTemp/Topling-1304093-Vrsutz 2024/11/21-03:38:15.208116 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099390.sst): first pass time = 1.05's, 128.091'MB/sec 2024/11/21-03:38:16.343958 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099390.sst): old prealloc_size = 147639500, real_size = 35760240 2024/11/21-03:38:16.344675 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099390.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927267 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 116.75 avg-zval = 36.40 2024/11/21-03:38:16.345498 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099390.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:38:17.182897 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099390.sst): second pass time = 1.14's, 95.374'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.656'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2558.904'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.14's, 294.051'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 19096.701'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.374'MB/sec zip pipeline throughput = 95.374'MB/sec entries = 927267 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.75 avg-zval = 36.40 usrkeys = 927267 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.75 avg-zval = 36.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927267 bytes = 3361360 } seq expand size = 6490869 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4471 value = 3.2072 dict = 1.00 all = 3.3572 } Zip/UnZip{ index = 0.0957 value = 0.3118 dict = 1.00 all = 0.2979 } ---------------------------- total value len = 564.889522 GB avg = 0.329 KB (by entry num) total key len = 44.729621 GB avg = 0.026 KB total ukey len = 33.090724 GB avg = 0.014 KB total ukey num = 2.335843663 Billion total entry num = 1.715345685 Billion write speed all = 17.683018769 MB/sec (with seq num) write speed all = 17.284967096 MB/sec (without seq num) 2024/11/21-03:38:17.232495 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99390: 927267 keys, 39979504 bytes, temperature: kWarm 2024/11/21-03:38:17.232568 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131497232535, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99390, "file_size": 39979504, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1080218058, "largest_seqno": 1484396437, "table_properties": {"data_size": 33753056, "index_size": 1775168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963476, "raw_average_key_size": 28, "raw_value_size": 108254354, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927267, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131494, "oldest_key_time": 0, "file_creation_time": 1732131494, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99390, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:38:17.268434 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099391.sst) /dev/shm/ToplingTemp/Topling-1304093-8ILMUy 2024/11/21-03:38:18.167487 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099391.sst): first pass time = 0.90's, 149.295'MB/sec 2024/11/21-03:38:19.289502 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099391.sst): old prealloc_size = 147639500, real_size = 35202256 2024/11/21-03:38:19.290119 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099391.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925817 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 116.97 avg-zval = 35.87 2024/11/21-03:38:19.290904 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099391.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:38:20.134976 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099391.sst): second pass time = 1.12's, 96.594'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.343'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3047.424'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.14's, 284.140'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34024.751'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 96.594'MB/sec zip pipeline throughput = 96.594'MB/sec entries = 925817 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 35.87 usrkeys = 925817 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 35.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925817 bytes = 3124640 } seq expand size = 6480719 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5138 value = 3.2610 dict = 1.00 all = 3.4255 } Zip/UnZip{ index = 0.0951 value = 0.3067 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 565.004298 GB avg = 0.329 KB (by entry num) total key len = 44.755544 GB avg = 0.026 KB total ukey len = 33.109240 GB avg = 0.014 KB total ukey num = 2.336769480 Billion total entry num = 1.716271502 Billion write speed all = 17.685585547 MB/sec (with seq num) write speed all = 17.287353136 MB/sec (without seq num) 2024/11/21-03:38:20.182328 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99391: 925817 keys, 39181754 bytes, temperature: kWarm 2024/11/21-03:38:20.182401 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131500182369, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99391, "file_size": 39181754, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1375097067, "largest_seqno": 1496200956, "table_properties": {"data_size": 33209472, "index_size": 1761152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25922876, "raw_average_key_size": 28, "raw_value_size": 108294906, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925817, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131497, "oldest_key_time": 0, "file_creation_time": 1732131497, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99391, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:38:20.219506 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099392.sst) /dev/shm/ToplingTemp/Topling-1304093-AMxviz 2024/11/21-03:38:21.172612 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099392.sst): first pass time = 0.95's, 140.828'MB/sec 2024/11/21-03:38:22.297512 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099392.sst): old prealloc_size = 147639500, real_size = 35606784 2024/11/21-03:38:22.298137 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099392.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927684 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0336 GB avg-val = 116.68 avg-zval = 36.23 2024/11/21-03:38:22.298876 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099392.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-03:38:23.138232 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099392.sst): second pass time = 1.12's, 96.304'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.955'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3002.310'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 297.016'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33590.681'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 96.304'MB/sec zip pipeline throughput = 96.304'MB/sec entries = 927684 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 36.23 usrkeys = 927684 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 36.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927684 bytes = 3014992 } seq expand size = 6493788 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5132 value = 3.2206 dict = 1.00 all = 3.4000 } Zip/UnZip{ index = 0.0951 value = 0.3105 dict = 1.00 all = 0.2941 } ---------------------------- total value len = 565.119034 GB avg = 0.329 KB (by entry num) total key len = 44.781519 GB avg = 0.026 KB total ukey len = 33.127794 GB avg = 0.014 KB total ukey num = 2.337697164 Billion total entry num = 1.717199186 Billion write speed all = 17.688126011 MB/sec (with seq num) write speed all = 17.289713051 MB/sec (without seq num) 2024/11/21-03:38:23.185222 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99392: 927684 keys, 39475702 bytes, temperature: kWarm 2024/11/21-03:38:23.185295 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131503185261, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99392, "file_size": 39475702, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1450135759, "largest_seqno": 1496201214, "table_properties": {"data_size": 33609904, "index_size": 1764800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975152, "raw_average_key_size": 28, "raw_value_size": 108242612, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927684, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131500, "oldest_key_time": 0, "file_creation_time": 1732131500, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99392, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:38:23.230372 140008660518464 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099393.sst) /dev/shm/ToplingTemp/Topling-1304093-RbV3HC 2024/11/21-03:38:23.320445 140008660518464 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099393.sst): first pass time = 0.09's, 254.161'MB/sec 2024/11/21-03:38:23.504078 140008660518464 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099393.sst): old prealloc_size = 147639500, real_size = 6879376 2024/11/21-03:38:23.504262 140008660518464 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099393.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 157987 prefix = 4 raw-key = 0.0025 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0185 GB zip-val = 0.0065 GB avg-val = 116.82 avg-zval = 41.37 2024/11/21-03:38:23.504455 140008660518464 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099393.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.023 GB 2024/11/21-03:38:23.642998 140008660518464 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00/099393.sst): second pass time = 0.18's, 100.954'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 54.467'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2123.146'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.340'MB/sec write SST data time = 0.02's, 332.692'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.193'MB, throughput = 7547.332'MB/sec zip my value time = 0.18's, unzip length = 0.018'GB zip my value throughput = 100.954'MB/sec zip pipeline throughput = 100.954'MB/sec entries = 157987 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.82 avg-zval = 41.37 usrkeys = 157987 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.82 avg-zval = 41.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 157987 bytes = 533216 } seq expand size = 1105909 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0032 GB value = 0.0185 GB dict = 0.19 MB all = 0.0229 GB } __ZipSize{ index = 0.0003 GB value = 0.0065 GB dict = 0.19 MB all = 0.0076 GB } UnZip/Zip{ index = 10.4048 value = 2.8238 dict = 1.00 all = 3.0233 } Zip/UnZip{ index = 0.0961 value = 0.3541 dict = 1.00 all = 0.3308 } ---------------------------- total value len = 565.138596 GB avg = 0.329 KB (by entry num) total key len = 44.785942 GB avg = 0.026 KB total ukey len = 33.130953 GB avg = 0.014 KB total ukey num = 2.337855151 Billion total entry num = 1.717357173 Billion write speed all = 17.688562702 MB/sec (with seq num) write speed all = 17.290118919 MB/sec (without seq num) 2024/11/21-03:38:23.658454 140008660518464 [compaction_job.cc:1948] [default] [JOB 4291] Generated table #99393: 157987 keys, 7567995 bytes, temperature: kWarm 2024/11/21-03:38:23.658518 140008660518464 EVENT_LOG_v1 {"time_micros": 1732131503658488, "cf_name": "default", "job": 4291, "event": "table_file_creation", "file_number": 99393, "file_size": 7567995, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1426792959, "largest_seqno": 1494720555, "table_properties": {"data_size": 6536000, "index_size": 303680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4423636, "raw_average_key_size": 28, "raw_value_size": 18456486, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 157987, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732131503, "oldest_key_time": 0, "file_creation_time": 1732131503, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "MNFBDNZRMES7C6T09KKW", "orig_file_number": 99393, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-03:38:23.669218 140008660518464 [compaction_job.cc:703] job-04291: subcompact[0], size: 0.717417 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04291/att-00 [099375,099376,099377,099378,099379,099380,099381,099382,099383,099384,099385,099386,099387,099388,099389,099390,099391,099392,099393] 2024/11/21-03:38:23.786097 140008660518464 (Original Log Time 2024/11/21-03:37:26.491267) EVENT_LOG_v1 {"time_micros": 1732131446491220, "job": 4291, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [99368, 99366], "files_L3": [99300, 99301, 99302, 99303, 99304, 99305, 99306, 99307, 99308, 99309, 99310, 99311, 99312, 99313, 99314, 99315, 99316, 99324], "score": 0.991287, "input_data_size": 743534623, "oldest_snapshot_seqno": 1496250528} 2024/11/21-03:38:23.786101 140008660518464 (Original Log Time 2024/11/21-03:38:23.784415) [compaction_job.cc:2022] [default] [JOB 4291] Compacted 2@2 + 18@3 files to L3 => 717417181 bytes 2024/11/21-03:38:23.786102 140008660518464 (Original Log Time 2024/11/21-03:38:23.786012) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 19 0 0 0] max score 0.03, MB/sec: 13.0 rd, 12.5 wr, level 3, files in(2, 18) out(19 +0 blob) MB in(31.1, 678.0 +0.0 blob) out(684.2 +0.0 blob), read-write-amplify(44.9) write-amplify(22.0) OK, records in: 16950005, records dropped: 109430 output_compression: Snappy 2024/11/21-03:38:23.786107 140008660518464 (Original Log Time 2024/11/21-03:38:23.786075) EVENT_LOG_v1 {"time_micros": 1732131503786035, "cf": "default", "job": 4291, "event": "compaction_finished", "compaction_time_micros": 57177959, "compaction_time_cpu_micros": 25913031, "output_level": 3, "num_output_files": 19, "total_output_size": 717417181, "num_input_records": 16950005, "num_output_records": 16840575, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 526526019, "file_range_sync_nanos": 0, "file_fsync_nanos": 16352, "file_prepare_write_nanos": 51132, "lsm_state": [0, 0, 0, 19, 0, 0, 0]} 2024/11/21-03:38:23.925305 140008660518464 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732121803,"starts":1732131446,"executesMs":57436,"compactionJobId":4291,"attempt":0,"compactionInputRawBytes":2454339582,"compactionInputZipBytes":743534623,"compactionOutputRawBytes":0,"compactionOutputZipBytes":717417181,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-3905a90ba2afcba2a56ddacdf64bfa2d-2e27880a09516f1d-00"}}, response = 2024/11/21-03:38:23.925414 140008660518464 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-21T00.56.43/.rocksdb/job-04291/att-00: olev 3, work 57.436 s, result 7.669 ms, install 1.707 ms, input{raw 2.286 GiB zip 709.090 MiB}