2024/11/20-15:36:30.213474 140257844221504 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00: fork to child time = 0.009648 sec 2024/11/20-15:36:30.213987 140257844221504 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00 2024/11/20-15:36:30.220990 140257844221504 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00 2024/11/20-15:36:30.222850 140257844221504 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-15:36:30.223087 140257844221504 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-15:36:30.270340 140257844221504 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-13225/att-00/MANIFEST-019309 2024/11/20-15:36:30.306128 140257844221504 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-13225/att-00/MANIFEST-019309 succeeded,manifest_file_number is 19309, next_file_number is 40698, last_sequence is 716860265, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 40497 2024/11/20-15:36:30.306136 140257844221504 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-15:36:30.306225 140257844221504 [version_set.cc:5724] Creating manifest 40698 2024/11/20-15:36:30.339689 140257844221504 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-15:36:30.364156 140257844221504 [compaction_job.cc:2437] [default] [JOB 13225] Compacting 1@2 + 10@3 files to L3, score 0.91, subcompactions 1 : 1 2024/11/20-15:36:30.364175 140257844221504 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 10621 Base level 2, inputs: [40696(27MB)], [40615(37MB) 40616(38MB) 40617(38MB) 40618(38MB) 40619(38MB) 40620(37MB) 40621(38MB) 40622(37MB) 40623(37MB) 40676(37MB)] 2024/11/20-15:36:30.371251 140257844221504 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-15:36:30.371274 140257844221504 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-15:36:30.371294 140257844221504 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-15:36:30.371302 140257844221504 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-15:36:30.371313 140257844221504 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-15:36:30.927031 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040699.sst) /dev/shm/ToplingTemp/Topling-494493-E6u92p 2024/11/20-15:36:31.664815 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040699.sst): first pass time = 0.74's, 181.939'MB/sec 2024/11/20-15:36:33.650135 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040699.sst): old prealloc_size = 147639500, real_size = 35272304 2024/11/20-15:36:33.650753 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040699.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921664 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.63 avg-zval = 36.08 2024/11/20-15:36:33.651631 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040699.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:36:34.685155 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040699.sst): second pass time = 1.98's, 54.727'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 38.030'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2976.464'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.23's, 172.676'MB/sec dict compress time = 4333933087.10's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 11982.026'MB/sec zip my value time = 1.98's, unzip length = 0.108'GB zip my value throughput = 54.727'MB/sec zip pipeline throughput = 179.248'MB/sec entries = 921664 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.63 avg-zval = 36.08 usrkeys = 921664 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.63 avg-zval = 36.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921664 bytes = 3225840 } seq expand size = 6451648 multi value expand size = 0 cnt WriteAppend = 28 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.3096 value = 3.2601 dict = 1.00 all = 3.4099 } Zip/UnZip{ index = 0.0970 value = 0.3067 dict = 1.00 all = 0.2933 } ---------------------------- total value len = 0.114863 GB avg = 0.125 KB (by entry num) total key len = 0.025807 GB avg = 0.028 KB total ukey len = 0.018433 GB avg = 0.020 KB total ukey num = 0.000921664 Billion total entry num = 0.000921664 Billion write speed all = 0.005733920 MB/sec (with seq num) write speed all = 0.005433372 MB/sec (without seq num) 2024/11/20-15:36:34.735633 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40699: 921664 keys, 39361014 bytes, temperature: kWarm 2024/11/20-15:36:34.735754 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088194735694, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40699, "file_size": 39361014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 522320272, "largest_seqno": 715940890, "table_properties": {"data_size": 33253728, "index_size": 1787968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25806592, "raw_average_key_size": 28, "raw_value_size": 108411144, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921664, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088190, "oldest_key_time": 0, "file_creation_time": 1732088190, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40699, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:36:34.806363 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040700.sst) /dev/shm/ToplingTemp/Topling-494493-8m215o 2024/11/20-15:36:35.959146 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040700.sst): first pass time = 1.15's, 116.435'MB/sec 2024/11/20-15:36:37.232559 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040700.sst): old prealloc_size = 147639500, real_size = 35705504 2024/11/20-15:36:37.233116 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040700.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922733 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0337 GB avg-val = 117.46 avg-zval = 36.53 2024/11/20-15:36:37.233900 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040700.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:36:38.148345 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040700.sst): second pass time = 1.27's, 85.153'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 43.453'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3290.731'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 318.214'MB/sec dict compress time = 360288524.20's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32932.908'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.153'MB/sec zip pipeline throughput = 85.324'MB/sec entries = 922733 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.46 avg-zval = 36.53 usrkeys = 922733 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.46 avg-zval = 36.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922733 bytes = 3229584 } seq expand size = 6459131 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.4446 value = 3.2153 dict = 1.00 all = 3.3729 } Zip/UnZip{ index = 0.0957 value = 0.3110 dict = 1.00 all = 0.2965 } ---------------------------- total value len = 0.229703 GB avg = 0.125 KB (by entry num) total key len = 0.051643 GB avg = 0.028 KB total ukey len = 0.036888 GB avg = 0.020 KB total ukey num = 0.001844397 Billion total entry num = 0.001844397 Billion write speed all = 0.011466527 MB/sec (with seq num) write speed all = 0.010865166 MB/sec (without seq num) 2024/11/20-15:36:38.197682 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40700: 922733 keys, 39793058 bytes, temperature: kWarm 2024/11/20-15:36:38.197749 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088198197716, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40700, "file_size": 39793058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 538603307, "largest_seqno": 715941708, "table_properties": {"data_size": 33707728, "index_size": 1766912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25836524, "raw_average_key_size": 28, "raw_value_size": 108381218, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922733, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088194, "oldest_key_time": 0, "file_creation_time": 1732088194, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40700, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:36:38.238253 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040701.sst) /dev/shm/ToplingTemp/Topling-494493-Df6Q5s 2024/11/20-15:36:39.487126 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040701.sst): first pass time = 1.25's, 107.476'MB/sec 2024/11/20-15:36:40.671909 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040701.sst): old prealloc_size = 147639500, real_size = 36445392 2024/11/20-15:36:40.672492 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040701.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923821 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0345 GB avg-val = 117.29 avg-zval = 37.30 2024/11/20-15:36:40.673340 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040701.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:36:41.506275 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040701.sst): second pass time = 1.18's, 91.527'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.583'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3220.228'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.12's, 326.498'MB/sec dict compress time = 360288524.20's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 17263.576'MB/sec zip my value time = 1.18's, unzip length = 0.108'GB zip my value throughput = 91.527'MB/sec zip pipeline throughput = 91.527'MB/sec entries = 923821 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.29 avg-zval = 37.30 usrkeys = 923821 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.29 avg-zval = 37.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923821 bytes = 3233392 } seq expand size = 6466747 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.5125 value = 3.1445 dict = 1.00 all = 3.3111 } Zip/UnZip{ index = 0.0951 value = 0.3180 dict = 1.00 all = 0.3020 } ---------------------------- total value len = 0.344521 GB avg = 0.124 KB (by entry num) total key len = 0.077510 GB avg = 0.028 KB total ukey len = 0.055364 GB avg = 0.020 KB total ukey num = 0.002768218 Billion total entry num = 0.002768218 Billion write speed all = 0.017197903 MB/sec (with seq num) write speed all = 0.016295456 MB/sec (without seq num) 2024/11/20-15:36:41.555841 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40701: 923821 keys, 40536126 bytes, temperature: kWarm 2024/11/20-15:36:41.555901 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088201555871, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40701, "file_size": 40536126, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 538603383, "largest_seqno": 715941492, "table_properties": {"data_size": 34456704, "index_size": 1757568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25866988, "raw_average_key_size": 28, "raw_value_size": 108350822, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923821, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088198, "oldest_key_time": 0, "file_creation_time": 1732088198, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40701, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:36:41.593081 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040702.sst) /dev/shm/ToplingTemp/Topling-494493-jFwl5s 2024/11/20-15:36:42.770412 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040702.sst): first pass time = 1.18's, 114.006'MB/sec 2024/11/20-15:36:43.930901 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040702.sst): old prealloc_size = 147639500, real_size = 35752528 2024/11/20-15:36:43.931679 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040702.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923051 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0337 GB avg-val = 117.41 avg-zval = 36.54 2024/11/20-15:36:43.932582 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040702.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:36:44.983268 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040702.sst): second pass time = 1.16's, 93.458'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 37.863'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2379.551'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.22's, 177.622'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 19078.156'MB/sec zip my value time = 1.16's, unzip length = 0.108'GB zip my value throughput = 93.458'MB/sec zip pipeline throughput = 93.458'MB/sec entries = 923051 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.41 avg-zval = 36.54 usrkeys = 923051 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.41 avg-zval = 36.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923051 bytes = 3230688 } seq expand size = 6461357 multi value expand size = 0 cnt WriteAppend = 57 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.3082 value = 3.2129 dict = 1.00 all = 3.3685 } Zip/UnZip{ index = 0.0970 value = 0.3112 dict = 1.00 all = 0.2969 } ---------------------------- total value len = 0.459354 GB avg = 0.124 KB (by entry num) total key len = 0.103356 GB avg = 0.028 KB total ukey len = 0.073825 GB avg = 0.020 KB total ukey num = 0.003691269 Billion total entry num = 0.003691269 Billion write speed all = 0.022927376 MB/sec (with seq num) write speed all = 0.021724183 MB/sec (without seq num) 2024/11/20-15:36:45.033410 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40702: 923051 keys, 39844902 bytes, temperature: kWarm 2024/11/20-15:36:45.033496 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088205033450, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40702, "file_size": 39844902, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 503040251, "largest_seqno": 715941012, "table_properties": {"data_size": 33730688, "index_size": 1790912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25845428, "raw_average_key_size": 28, "raw_value_size": 108372314, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923051, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088201, "oldest_key_time": 0, "file_creation_time": 1732088201, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40702, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:36:45.107647 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040703.sst) /dev/shm/ToplingTemp/Topling-494493-8M5MWr 2024/11/20-15:36:46.073643 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040703.sst): first pass time = 0.97's, 138.950'MB/sec 2024/11/20-15:36:47.218722 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040703.sst): old prealloc_size = 147639500, real_size = 36413120 2024/11/20-15:36:47.219363 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040703.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925827 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.97 avg-zval = 37.17 2024/11/20-15:36:47.220147 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040703.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:36:48.263988 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040703.sst): second pass time = 1.14's, 94.639'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 38.645'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2855.906'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.26's, 158.352'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 17456.846'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 94.639'MB/sec zip pipeline throughput = 94.639'MB/sec entries = 925827 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.97 avg-zval = 37.17 usrkeys = 925827 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.97 avg-zval = 37.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925827 bytes = 3124688 } seq expand size = 6480789 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.4456 value = 3.1473 dict = 1.00 all = 3.3228 } Zip/UnZip{ index = 0.0957 value = 0.3177 dict = 1.00 all = 0.3009 } ---------------------------- total value len = 0.574130 GB avg = 0.124 KB (by entry num) total key len = 0.129279 GB avg = 0.028 KB total ukey len = 0.092342 GB avg = 0.020 KB total ukey num = 0.004617096 Billion total entry num = 0.004617096 Billion write speed all = 0.028656250 MB/sec (with seq num) write speed all = 0.027151478 MB/sec (without seq num) 2024/11/20-15:36:48.313823 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40703: 925827 keys, 40392490 bytes, temperature: kWarm 2024/11/20-15:36:48.313914 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088208313869, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40703, "file_size": 40392490, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 591115808, "largest_seqno": 715941624, "table_properties": {"data_size": 34408816, "index_size": 1772672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923156, "raw_average_key_size": 28, "raw_value_size": 108294634, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925827, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088205, "oldest_key_time": 0, "file_creation_time": 1732088205, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40703, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:36:48.381058 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040704.sst) /dev/shm/ToplingTemp/Topling-494493-MhQbit 2024/11/20-15:36:49.274460 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040704.sst): first pass time = 0.89's, 150.242'MB/sec 2024/11/20-15:36:50.418198 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040704.sst): old prealloc_size = 147639500, real_size = 35239168 2024/11/20-15:36:50.418780 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040704.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921904 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.59 avg-zval = 36.07 2024/11/20-15:36:50.419585 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040704.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:36:51.555334 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040704.sst): second pass time = 1.14's, 94.850'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.14's, 34.481'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3133.320'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.249'MB/sec write SST data time = 0.21's, 185.430'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 18944.515'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 94.850'MB/sec zip pipeline throughput = 94.850'MB/sec entries = 921904 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.59 avg-zval = 36.07 usrkeys = 921904 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.59 avg-zval = 36.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921904 bytes = 3111440 } seq expand size = 6453328 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5098 value = 3.2599 dict = 1.00 all = 3.4231 } Zip/UnZip{ index = 0.0951 value = 0.3068 dict = 1.00 all = 0.2921 } ---------------------------- total value len = 0.688988 GB avg = 0.124 KB (by entry num) total key len = 0.155092 GB avg = 0.028 KB total ukey len = 0.110780 GB avg = 0.020 KB total ukey num = 0.005539000 Billion total entry num = 0.005539000 Billion write speed all = 0.034382456 MB/sec (with seq num) write speed all = 0.032577466 MB/sec (without seq num) 2024/11/20-15:36:51.602193 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40704: 921904 keys, 39209348 bytes, temperature: kWarm 2024/11/20-15:36:51.602261 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088211602230, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40704, "file_size": 39209348, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 591115820, "largest_seqno": 715940136, "table_properties": {"data_size": 33254128, "index_size": 1754368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25813312, "raw_average_key_size": 28, "raw_value_size": 108404420, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921904, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088208, "oldest_key_time": 0, "file_creation_time": 1732088208, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40704, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:36:51.637469 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040705.sst) /dev/shm/ToplingTemp/Topling-494493-5bCx8s 2024/11/20-15:36:52.465439 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040705.sst): first pass time = 0.83's, 162.112'MB/sec 2024/11/20-15:36:53.689131 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040705.sst): old prealloc_size = 147639500, real_size = 36586736 2024/11/20-15:36:53.689667 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040705.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924075 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0346 GB avg-val = 117.25 avg-zval = 37.40 2024/11/20-15:36:53.690518 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040705.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:36:54.686524 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040705.sst): second pass time = 1.22's, 88.595'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 40.904'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3502.516'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.266'MB/sec write SST data time = 0.13's, 319.987'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32616.540'MB/sec zip my value time = 1.22's, unzip length = 0.108'GB zip my value throughput = 88.595'MB/sec zip pipeline throughput = 88.595'MB/sec entries = 924075 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.25 avg-zval = 37.40 usrkeys = 924075 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.25 avg-zval = 37.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924075 bytes = 3349792 } seq expand size = 6468525 multi value expand size = 0 cnt WriteAppend = 63 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0408 GB } UnZip/Zip{ index = 10.3056 value = 3.1347 dict = 1.00 all = 3.2899 } Zip/UnZip{ index = 0.0970 value = 0.3190 dict = 1.00 all = 0.3040 } ---------------------------- total value len = 0.803800 GB avg = 0.124 KB (by entry num) total key len = 0.180966 GB avg = 0.028 KB total ukey len = 0.129262 GB avg = 0.020 KB total ukey num = 0.006463075 Billion total entry num = 0.006463075 Billion write speed all = 0.040108013 MB/sec (with seq num) write speed all = 0.038002164 MB/sec (without seq num) 2024/11/20-15:36:54.734876 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40705: 924075 keys, 40796754 bytes, temperature: kWarm 2024/11/20-15:36:54.734937 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088214734908, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40705, "file_size": 40796754, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 431720469, "largest_seqno": 715937257, "table_properties": {"data_size": 34562208, "index_size": 1793344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25874100, "raw_average_key_size": 28, "raw_value_size": 108343770, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924075, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088211, "oldest_key_time": 0, "file_creation_time": 1732088211, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40705, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:36:54.770033 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040706.sst) /dev/shm/ToplingTemp/Topling-494493-mKBXRp 2024/11/20-15:36:55.678684 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040706.sst): first pass time = 0.91's, 147.734'MB/sec 2024/11/20-15:36:56.812079 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040706.sst): old prealloc_size = 147639500, real_size = 35148400 2024/11/20-15:36:56.812613 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040706.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925230 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 117.06 avg-zval = 35.84 2024/11/20-15:36:56.813439 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040706.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:36:57.704555 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040706.sst): second pass time = 1.13's, 95.627'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.970'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3437.195'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.302'MB/sec write SST data time = 0.12's, 314.390'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33588.241'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 95.627'MB/sec zip pipeline throughput = 95.627'MB/sec entries = 925230 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.06 avg-zval = 35.84 usrkeys = 925230 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.06 avg-zval = 35.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925230 bytes = 3238320 } seq expand size = 6476610 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5155 value = 3.2666 dict = 1.00 all = 3.4203 } Zip/UnZip{ index = 0.0951 value = 0.3061 dict = 1.00 all = 0.2924 } ---------------------------- total value len = 0.918588 GB avg = 0.124 KB (by entry num) total key len = 0.206873 GB avg = 0.028 KB total ukey len = 0.147766 GB avg = 0.020 KB total ukey num = 0.007388305 Billion total entry num = 0.007388305 Billion write speed all = 0.045832647 MB/sec (with seq num) write speed all = 0.043425628 MB/sec (without seq num) 2024/11/20-15:36:57.750841 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40706: 925230 keys, 39242068 bytes, temperature: kWarm 2024/11/20-15:36:57.750900 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088217750871, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40706, "file_size": 39242068, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 542657405, "largest_seqno": 715940690, "table_properties": {"data_size": 33157152, "index_size": 1759744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906440, "raw_average_key_size": 28, "raw_value_size": 108311348, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925230, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088214, "oldest_key_time": 0, "file_creation_time": 1732088214, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40706, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:36:57.786447 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040707.sst) /dev/shm/ToplingTemp/Topling-494493-ruJVRr 2024/11/20-15:36:58.680478 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040707.sst): first pass time = 0.89's, 150.132'MB/sec 2024/11/20-15:36:59.823397 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040707.sst): old prealloc_size = 147639500, real_size = 35166160 2024/11/20-15:36:59.823937 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040707.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925219 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 117.07 avg-zval = 35.84 2024/11/20-15:36:59.824843 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040707.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:37:00.714675 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040707.sst): second pass time = 1.14's, 94.843'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 44.050'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3448.313'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.302'MB/sec write SST data time = 0.13's, 311.459'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 19150.961'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 94.843'MB/sec zip pipeline throughput = 94.843'MB/sec entries = 925219 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.07 avg-zval = 35.84 usrkeys = 925219 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.07 avg-zval = 35.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925219 bytes = 3238288 } seq expand size = 6476533 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4436 value = 3.2661 dict = 1.00 all = 3.4187 } Zip/UnZip{ index = 0.0958 value = 0.3062 dict = 1.00 all = 0.2925 } ---------------------------- total value len = 1.033376 GB avg = 0.124 KB (by entry num) total key len = 0.232779 GB avg = 0.028 KB total ukey len = 0.166270 GB avg = 0.020 KB total ukey num = 0.008313524 Billion total entry num = 0.008313524 Billion write speed all = 0.051555888 MB/sec (with seq num) write speed all = 0.048847776 MB/sec (without seq num) 2024/11/20-15:37:00.761563 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40707: 925219 keys, 39259936 bytes, temperature: kWarm 2024/11/20-15:37:00.761622 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088220761591, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40707, "file_size": 39259936, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 542657474, "largest_seqno": 715940739, "table_properties": {"data_size": 33162816, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906132, "raw_average_key_size": 28, "raw_value_size": 108311658, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925219, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088217, "oldest_key_time": 0, "file_creation_time": 1732088217, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40707, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:37:00.796007 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040708.sst) /dev/shm/ToplingTemp/Topling-494493-zv07fs 2024/11/20-15:37:01.616790 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040708.sst): first pass time = 0.82's, 163.531'MB/sec 2024/11/20-15:37:02.816105 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040708.sst): old prealloc_size = 147639500, real_size = 35852624 2024/11/20-15:37:02.816626 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040708.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926156 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 116.92 avg-zval = 36.53 2024/11/20-15:37:02.817241 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040708.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:37:03.630697 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040708.sst): second pass time = 1.20's, 90.341'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 47.947'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3559.114'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 294.153'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27401.075'MB/sec zip my value time = 1.20's, unzip length = 0.108'GB zip my value throughput = 90.341'MB/sec zip pipeline throughput = 90.341'MB/sec entries = 926156 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.92 avg-zval = 36.53 usrkeys = 926156 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.67 avg-zval = 36.53 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 628801 bytes = 2200816 } seq expand size = 4401607 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.3758 value = 3.2003 dict = 1.00 all = 3.4365 } Zip/UnZip{ index = 0.0964 value = 0.3125 dict = 1.00 all = 0.2910 } ---------------------------- total value len = 1.146063 GB avg = 0.124 KB (by entry num) total key len = 0.258711 GB avg = 0.028 KB total ukey len = 0.184794 GB avg = 0.020 KB total ukey num = 0.009239680 Billion total entry num = 0.009239680 Billion write speed all = 0.057193468 MB/sec (with seq num) write speed all = 0.054184019 MB/sec (without seq num) 2024/11/20-15:37:03.678764 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40708: 926156 keys, 39056806 bytes, temperature: kWarm 2024/11/20-15:37:03.678818 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088223678787, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40708, "file_size": 39056806, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 715941740, "table_properties": {"data_size": 33835712, "index_size": 1785216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25932368, "raw_average_key_size": 28, "raw_value_size": 108285420, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926156, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088220, "oldest_key_time": 0, "file_creation_time": 1732088220, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40708, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:37:03.714160 140257844221504 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040709.sst) /dev/shm/ToplingTemp/Topling-494493-VCUuFs 2024/11/20-15:37:03.784395 140257844221504 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040709.sst): first pass time = 0.07's, 266.252'MB/sec 2024/11/20-15:37:03.919358 140257844221504 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040709.sst): old prealloc_size = 147639500, real_size = 5184672 2024/11/20-15:37:03.919497 140257844221504 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040709.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129180 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0151 GB zip-val = 0.0049 GB avg-val = 116.68 avg-zval = 37.97 2024/11/20-15:37:03.919540 140257844221504 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040709.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.019 GB 2024/11/20-15:37:04.017688 140257844221504 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00/040709.sst): second pass time = 0.13's, 112.276'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.10's, 54.021'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2216.311'MB/sec (index lex order gen) rebuild zvType time = 0.08's, 0.405'MB/sec write SST data time = 0.02's, 286.695'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.158'MB, throughput = 3088.699'MB/sec zip my value time = 0.13's, unzip length = 0.015'GB zip my value throughput = 112.276'MB/sec zip pipeline throughput = 112.276'MB/sec entries = 129180 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.68 avg-zval = 37.97 usrkeys = 129180 avg-key = 20.00 avg-zkey = 1.91 avg-val = 116.68 avg-zval = 37.97 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0026 GB value = 0.0151 GB dict = 0.16 MB all = 0.0187 GB } __ZipSize{ index = 0.0002 GB value = 0.0049 GB dict = 0.16 MB all = 0.0053 GB } UnZip/Zip{ index = 10.4474 value = 3.0730 dict = 1.00 all = 3.5184 } Zip/UnZip{ index = 0.0957 value = 0.3254 dict = 1.00 all = 0.2842 } ---------------------------- total value len = 1.161136 GB avg = 0.124 KB (by entry num) total key len = 0.262328 GB avg = 0.028 KB total ukey len = 0.187377 GB avg = 0.020 KB total ukey num = 0.009368860 Billion total entry num = 0.009368860 Billion write speed all = 0.057953476 MB/sec (with seq num) write speed all = 0.054902001 MB/sec (without seq num) 2024/11/20-15:37:04.030117 140257844221504 [compaction_job.cc:1948] [default] [JOB 13225] Generated table #40709: 129180 keys, 5311958 bytes, temperature: kWarm 2024/11/20-15:37:04.030175 140257844221504 EVENT_LOG_v1 {"time_micros": 1732088224030145, "cf_name": "default", "job": 13225, "event": "table_file_creation", "file_number": 40709, "file_size": 5311958, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 4904912, "index_size": 247296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3617040, "raw_average_key_size": 28, "raw_value_size": 15072556, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 129180, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732088223, "oldest_key_time": 0, "file_creation_time": 1732088223, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 40709, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:37:04.044483 140257844221504 [compaction_job.cc:703] job-13225: subcompact[0], size: 0.402804 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13225/att-00 [040699,040700,040701,040702,040703,040704,040705,040706,040707,040708,040709] 2024/11/20-15:37:04.113024 140257844221504 (Original Log Time 2024/11/20-15:36:30.364241) EVENT_LOG_v1 {"time_micros": 1732088190364193, "job": 13225, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [40696], "files_L3": [40615, 40616, 40617, 40618, 40619, 40620, 40621, 40622, 40623, 40676], "score": 0.912427, "input_data_size": 426055509, "oldest_snapshot_seqno": 716860166} 2024/11/20-15:37:04.113028 140257844221504 (Original Log Time 2024/11/20-15:37:04.111887) [compaction_job.cc:2022] [default] [JOB 13225] Compacted 1@2 + 10@3 files to L3 => 402804460 bytes 2024/11/20-15:37:04.113029 140257844221504 (Original Log Time 2024/11/20-15:37:04.112943) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.02, MB/sec: 12.7 rd, 12.0 wr, level 3, files in(1, 10) out(11 +0 blob) MB in(27.1, 379.2 +0.0 blob) out(384.1 +0.0 blob), read-write-amplify(29.2) write-amplify(14.2) OK, records in: 9473355, records dropped: 104495 output_compression: Snappy 2024/11/20-15:37:04.113033 140257844221504 (Original Log Time 2024/11/20-15:37:04.113003) EVENT_LOG_v1 {"time_micros": 1732088224112977, "cf": "default", "job": 13225, "event": "compaction_finished", "compaction_time_micros": 33680246, "compaction_time_cpu_micros": 15994955, "output_level": 3, "num_output_files": 11, "total_output_size": 402804460, "num_input_records": 9473355, "num_output_records": 9368860, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 330511314, "file_range_sync_nanos": 0, "file_fsync_nanos": 10335, "file_prepare_write_nanos": 31193, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/20-15:37:04.240464 140257844221504 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732076400,"starts":1732088190,"executesMs":33890,"compactionJobId":13225,"attempt":0,"compactionInputRawBytes":1375705674,"compactionInputZipBytes":426055509,"compactionOutputRawBytes":0,"compactionOutputZipBytes":402804460,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-ce23304a13af4f8ee199d781fa9d3342-5847a95b7b0e8add-00"}}, response = 2024/11/20-15:37:04.240534 140257844221504 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-13225/att-00: olev 3, work 33.890 s, result 7.854 ms, install 1.164 ms, input{raw 1.281 GiB zip 406.318 MiB}