2024/11/20-15:56:01.183945 140257533691456 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00: fork to child time = 0.009069 sec 2024/11/20-15:56:01.184505 140257533691456 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00 2024/11/20-15:56:01.191086 140257533691456 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00 2024/11/20-15:56:01.192724 140257533691456 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-15:56:01.192960 140257533691456 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-15:56:01.236769 140257533691456 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-14213/att-00/MANIFEST-019309 2024/11/20-15:56:01.286846 140257533691456 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-14213/att-00/MANIFEST-019309 succeeded,manifest_file_number is 19309, next_file_number is 42523, last_sequence is 739349759, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 42287 2024/11/20-15:56:01.286852 140257533691456 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-15:56:01.286951 140257533691456 [version_set.cc:5724] Creating manifest 42523 2024/11/20-15:56:01.317938 140257533691456 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-15:56:01.337730 140257533691456 [compaction_job.cc:2437] [default] [JOB 14213] Compacting 1@2 + 12@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/20-15:56:01.337741 140257533691456 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 11260 Base level 2, inputs: [42518(27MB)], [42350(36MB) 42351(37MB) 42352(38MB) 42353(37MB) 42498(37MB) 42499(37MB) 42500(37MB) 42501(38MB) 42502(36MB) 42503(37MB) 42504(37MB) 42505(38MB)] 2024/11/20-15:56:01.343728 140257533691456 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-15:56:01.343752 140257533691456 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-15:56:01.343773 140257533691456 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-15:56:01.343781 140257533691456 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-15:56:01.343790 140257533691456 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-15:56:01.865705 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042524.sst) /dev/shm/ToplingTemp/Topling-527435-WfUldo 2024/11/20-15:56:02.357650 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042524.sst): first pass time = 0.49's, 272.858'MB/sec 2024/11/20-15:56:03.462988 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042524.sst): old prealloc_size = 147639500, real_size = 34483360 2024/11/20-15:56:03.463560 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042524.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922038 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0325 GB avg-val = 117.57 avg-zval = 35.21 2024/11/20-15:56:03.464378 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042524.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:04.299114 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042524.sst): second pass time = 1.10's, 98.151'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.133'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3262.996'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.12's, 315.593'MB/sec dict compress time = 8314045560.10's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33065.808'MB/sec zip my value time = 1.10's, unzip length = 0.108'GB zip my value throughput = 98.151'MB/sec zip pipeline throughput = 98.151'MB/sec entries = 922038 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.57 avg-zval = 35.21 usrkeys = 922038 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.57 avg-zval = 35.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922038 bytes = 3227152 } seq expand size = 6454266 multi value expand size = 0 cnt WriteAppend = 27 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.3083 value = 3.3391 dict = 1.00 all = 3.4796 } Zip/UnZip{ index = 0.0970 value = 0.2995 dict = 1.00 all = 0.2874 } ---------------------------- total value len = 0.114855 GB avg = 0.125 KB (by entry num) total key len = 0.025817 GB avg = 0.028 KB total ukey len = 0.018441 GB avg = 0.020 KB total ukey num = 0.000922038 Billion total entry num = 0.000922038 Billion write speed all = 0.005473097 MB/sec (with seq num) write speed all = 0.005186109 MB/sec (without seq num) 2024/11/20-15:56:04.348706 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42524: 922038 keys, 38572790 bytes, temperature: kWarm 2024/11/20-15:56:04.348778 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089364348742, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42524, "file_size": 38572790, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 522320272, "largest_seqno": 738253674, "table_properties": {"data_size": 32463760, "index_size": 1788928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25817064, "raw_average_key_size": 28, "raw_value_size": 108400728, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922038, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089361, "oldest_key_time": 0, "file_creation_time": 1732089361, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42524, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:04.383335 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042525.sst) /dev/shm/ToplingTemp/Topling-527435-4xlOVk 2024/11/20-15:56:05.187880 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042525.sst): first pass time = 0.80's, 166.830'MB/sec 2024/11/20-15:56:06.329377 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042525.sst): old prealloc_size = 147639500, real_size = 34844384 2024/11/20-15:56:06.329937 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042525.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921921 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0328 GB avg-val = 117.58 avg-zval = 35.63 2024/11/20-15:56:06.330739 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042525.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:07.165210 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042525.sst): second pass time = 1.14's, 95.033'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.577'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3299.795'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.12's, 315.969'MB/sec dict compress time = 0.00's, 1082565.435'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31922.701'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.033'MB/sec zip pipeline throughput = 95.033'MB/sec entries = 921921 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.58 avg-zval = 35.63 usrkeys = 921921 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.58 avg-zval = 35.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921921 bytes = 3226736 } seq expand size = 6453447 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4460 value = 3.3001 dict = 1.00 all = 3.4477 } Zip/UnZip{ index = 0.0957 value = 0.3030 dict = 1.00 all = 0.2901 } ---------------------------- total value len = 0.229712 GB avg = 0.125 KB (by entry num) total key len = 0.051631 GB avg = 0.028 KB total ukey len = 0.036879 GB avg = 0.020 KB total ukey num = 0.001843959 Billion total entry num = 0.001843959 Billion write speed all = 0.010944943 MB/sec (with seq num) write speed all = 0.010371067 MB/sec (without seq num) 2024/11/20-15:56:07.213411 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42525: 921921 keys, 38930064 bytes, temperature: kWarm 2024/11/20-15:56:07.213474 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089367213446, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42525, "file_size": 38930064, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 538603307, "largest_seqno": 738252987, "table_properties": {"data_size": 32848592, "index_size": 1765120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25813788, "raw_average_key_size": 28, "raw_value_size": 108404042, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921921, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089364, "oldest_key_time": 0, "file_creation_time": 1732089364, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42525, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:07.248230 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042526.sst) /dev/shm/ToplingTemp/Topling-527435-WG28xn 2024/11/20-15:56:08.058665 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042526.sst): first pass time = 0.81's, 165.619'MB/sec 2024/11/20-15:56:09.157942 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042526.sst): old prealloc_size = 147639500, real_size = 35814752 2024/11/20-15:56:09.158513 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042526.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923239 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0338 GB avg-val = 117.38 avg-zval = 36.64 2024/11/20-15:56:09.159296 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042526.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:09.980146 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042526.sst): second pass time = 1.10's, 98.657'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.394'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3211.308'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.12's, 320.696'MB/sec dict compress time = 140278.09's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34213.322'MB/sec zip my value time = 1.10's, unzip length = 0.108'GB zip my value throughput = 98.657'MB/sec zip pipeline throughput = 98.657'MB/sec entries = 923239 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.38 avg-zval = 36.64 usrkeys = 923239 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.38 avg-zval = 36.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923239 bytes = 3115952 } seq expand size = 6462673 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.5120 value = 3.2035 dict = 1.00 all = 3.3732 } Zip/UnZip{ index = 0.0951 value = 0.3122 dict = 1.00 all = 0.2965 } ---------------------------- total value len = 0.344542 GB avg = 0.125 KB (by entry num) total key len = 0.077482 GB avg = 0.028 KB total ukey len = 0.055344 GB avg = 0.020 KB total ukey num = 0.002767198 Billion total entry num = 0.002767198 Billion write speed all = 0.016415961 MB/sec (with seq num) write speed all = 0.015554849 MB/sec (without seq num) 2024/11/20-15:56:10.028934 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42526: 923239 keys, 39788908 bytes, temperature: kWarm 2024/11/20-15:56:10.028995 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089370028965, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42526, "file_size": 39788908, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 660052313, "largest_seqno": 738251740, "table_properties": {"data_size": 33827216, "index_size": 1756544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25850692, "raw_average_key_size": 28, "raw_value_size": 108367150, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923239, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089367, "oldest_key_time": 0, "file_creation_time": 1732089367, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42526, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:10.064429 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042527.sst) /dev/shm/ToplingTemp/Topling-527435-YtQsnm 2024/11/20-15:56:10.884558 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042527.sst): first pass time = 0.82's, 163.660'MB/sec 2024/11/20-15:56:11.989891 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042527.sst): old prealloc_size = 147639500, real_size = 35694560 2024/11/20-15:56:11.990477 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042527.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923376 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0337 GB avg-val = 117.36 avg-zval = 36.47 2024/11/20-15:56:11.991238 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042527.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:12.815293 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042527.sst): second pass time = 1.10's, 98.108'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.205'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.584'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.12's, 322.812'MB/sec dict compress time = 140278.09's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33566.007'MB/sec zip my value time = 1.10's, unzip length = 0.108'GB zip my value throughput = 98.108'MB/sec zip pipeline throughput = 98.108'MB/sec entries = 923376 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.36 avg-zval = 36.47 usrkeys = 923376 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.36 avg-zval = 36.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923376 bytes = 3231824 } seq expand size = 6463632 multi value expand size = 0 cnt WriteAppend = 56 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.3081 value = 3.2182 dict = 1.00 all = 3.3733 } Zip/UnZip{ index = 0.0970 value = 0.3107 dict = 1.00 all = 0.2964 } ---------------------------- total value len = 0.459369 GB avg = 0.124 KB (by entry num) total key len = 0.103336 GB avg = 0.028 KB total ukey len = 0.073811 GB avg = 0.020 KB total ukey num = 0.003690574 Billion total entry num = 0.003690574 Billion write speed all = 0.021885800 MB/sec (with seq num) write speed all = 0.020737474 MB/sec (without seq num) 2024/11/20-15:56:12.864222 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42527: 923376 keys, 39787742 bytes, temperature: kWarm 2024/11/20-15:56:12.864274 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089372864248, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42527, "file_size": 39787742, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 503040251, "largest_seqno": 738252280, "table_properties": {"data_size": 33671952, "index_size": 1791552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25854528, "raw_average_key_size": 28, "raw_value_size": 108363296, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923376, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089370, "oldest_key_time": 0, "file_creation_time": 1732089370, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42527, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:12.899218 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042528.sst) /dev/shm/ToplingTemp/Topling-527435-6737so 2024/11/20-15:56:13.715209 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042528.sst): first pass time = 0.82's, 164.491'MB/sec 2024/11/20-15:56:14.841415 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042528.sst): old prealloc_size = 147639500, real_size = 35177600 2024/11/20-15:56:14.841978 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042528.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923428 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.35 avg-zval = 35.94 2024/11/20-15:56:14.842767 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042528.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:15.671187 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042528.sst): second pass time = 1.13's, 96.286'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.324'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3262.022'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.12's, 320.843'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35689.838'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 96.286'MB/sec zip pipeline throughput = 96.286'MB/sec entries = 923428 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.35 avg-zval = 35.94 usrkeys = 923428 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.35 avg-zval = 35.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923428 bytes = 3232016 } seq expand size = 6463996 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5130 value = 3.2649 dict = 1.00 all = 3.4180 } Zip/UnZip{ index = 0.0951 value = 0.3063 dict = 1.00 all = 0.2926 } ---------------------------- total value len = 0.574195 GB avg = 0.124 KB (by entry num) total key len = 0.129192 GB avg = 0.028 KB total ukey len = 0.092280 GB avg = 0.020 KB total ukey num = 0.004614002 Billion total entry num = 0.004614002 Billion write speed all = 0.027354424 MB/sec (with seq num) write speed all = 0.025918932 MB/sec (without seq num) 2024/11/20-15:56:15.719016 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42528: 923428 keys, 39267386 bytes, temperature: kWarm 2024/11/20-15:56:15.719097 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089375719065, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42528, "file_size": 39267386, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 591115717, "largest_seqno": 738251359, "table_properties": {"data_size": 33189808, "index_size": 1756736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25855984, "raw_average_key_size": 28, "raw_value_size": 108361800, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089372, "oldest_key_time": 0, "file_creation_time": 1732089372, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42528, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:15.755626 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042529.sst) /dev/shm/ToplingTemp/Topling-527435-FJAKYn 2024/11/20-15:56:16.568505 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042529.sst): first pass time = 0.81's, 165.120'MB/sec 2024/11/20-15:56:17.680338 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042529.sst): old prealloc_size = 147639500, real_size = 35790624 2024/11/20-15:56:17.680863 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042529.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923484 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0338 GB avg-val = 117.34 avg-zval = 36.60 2024/11/20-15:56:17.681650 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042529.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:18.501328 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042529.sst): second pass time = 1.11's, 97.536'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.436'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3514.639'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.12's, 327.594'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 19309.330'MB/sec zip my value time = 1.11's, unzip length = 0.108'GB zip my value throughput = 97.536'MB/sec zip pipeline throughput = 97.536'MB/sec entries = 923484 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.34 avg-zval = 36.60 usrkeys = 923484 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.34 avg-zval = 36.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923484 bytes = 3116768 } seq expand size = 6464388 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.5136 value = 3.2057 dict = 1.00 all = 3.3753 } Zip/UnZip{ index = 0.0951 value = 0.3119 dict = 1.00 all = 0.2963 } ---------------------------- total value len = 0.689020 GB avg = 0.124 KB (by entry num) total key len = 0.155050 GB avg = 0.028 KB total ukey len = 0.110750 GB avg = 0.020 KB total ukey num = 0.005537486 Billion total entry num = 0.005537486 Billion write speed all = 0.032821881 MB/sec (with seq num) write speed all = 0.031099267 MB/sec (without seq num) 2024/11/20-15:56:18.550283 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42529: 923484 keys, 39765110 bytes, temperature: kWarm 2024/11/20-15:56:18.550341 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089378550312, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42529, "file_size": 39765110, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 625722464, "largest_seqno": 738253269, "table_properties": {"data_size": 33802832, "index_size": 1756736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25857552, "raw_average_key_size": 28, "raw_value_size": 108360216, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923484, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089375, "oldest_key_time": 0, "file_creation_time": 1732089375, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42529, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:18.584792 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042530.sst) /dev/shm/ToplingTemp/Topling-527435-UGgdHn 2024/11/20-15:56:19.383071 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042530.sst): first pass time = 0.80's, 168.140'MB/sec 2024/11/20-15:56:20.501357 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042530.sst): old prealloc_size = 147639500, real_size = 34884352 2024/11/20-15:56:20.501883 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042530.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922318 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0329 GB avg-val = 117.52 avg-zval = 35.67 2024/11/20-15:56:20.502695 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042530.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:21.331699 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042530.sst): second pass time = 1.12's, 96.998'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 46.934'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3489.223'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.12's, 321.389'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 17188.233'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 96.998'MB/sec zip pipeline throughput = 96.998'MB/sec entries = 922318 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 35.67 usrkeys = 922318 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.52 avg-zval = 35.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922318 bytes = 3228128 } seq expand size = 6456226 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5103 value = 3.2948 dict = 1.00 all = 3.4440 } Zip/UnZip{ index = 0.0951 value = 0.3035 dict = 1.00 all = 0.2904 } ---------------------------- total value len = 0.803869 GB avg = 0.124 KB (by entry num) total key len = 0.180875 GB avg = 0.028 KB total ukey len = 0.129196 GB avg = 0.020 KB total ukey num = 0.006459804 Billion total entry num = 0.006459804 Billion write speed all = 0.038287819 MB/sec (with seq num) write speed all = 0.036278509 MB/sec (without seq num) 2024/11/20-15:56:21.378494 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42530: 922318 keys, 38971014 bytes, temperature: kWarm 2024/11/20-15:56:21.378545 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089381378517, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42530, "file_size": 38971014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 591115808, "largest_seqno": 738251445, "table_properties": {"data_size": 32898544, "index_size": 1755072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25824904, "raw_average_key_size": 28, "raw_value_size": 108392932, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922318, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089378, "oldest_key_time": 0, "file_creation_time": 1732089378, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42530, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:21.416861 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042531.sst) /dev/shm/ToplingTemp/Topling-527435-IFCPUk 2024/11/20-15:56:22.221085 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042531.sst): first pass time = 0.80's, 166.897'MB/sec 2024/11/20-15:56:23.405039 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042531.sst): old prealloc_size = 147639500, real_size = 35772448 2024/11/20-15:56:23.405574 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042531.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925420 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0337 GB avg-val = 117.03 avg-zval = 36.46 2024/11/20-15:56:23.406423 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042531.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:24.167220 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042531.sst): second pass time = 1.18's, 91.545'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 52.463'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3660.419'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.362'MB/sec write SST data time = 0.12's, 326.999'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 16126.807'MB/sec zip my value time = 1.18's, unzip length = 0.108'GB zip my value throughput = 91.545'MB/sec zip pipeline throughput = 91.545'MB/sec entries = 925420 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.03 avg-zval = 36.46 usrkeys = 925420 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.03 avg-zval = 36.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925420 bytes = 3354656 } seq expand size = 6477940 multi value expand size = 0 cnt WriteAppend = 56 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.3059 value = 3.2095 dict = 1.00 all = 3.3566 } Zip/UnZip{ index = 0.0970 value = 0.3116 dict = 1.00 all = 0.2979 } ---------------------------- total value len = 0.918653 GB avg = 0.124 KB (by entry num) total key len = 0.206786 GB avg = 0.028 KB total ukey len = 0.147704 GB avg = 0.020 KB total ukey num = 0.007385224 Billion total entry num = 0.007385224 Billion write speed all = 0.043753388 MB/sec (with seq num) write speed all = 0.041456481 MB/sec (without seq num) 2024/11/20-15:56:24.214531 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42531: 925420 keys, 39985842 bytes, temperature: kWarm 2024/11/20-15:56:24.214583 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089384214556, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42531, "file_size": 39985842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 431720469, "largest_seqno": 738253074, "table_properties": {"data_size": 33744976, "index_size": 1795904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911760, "raw_average_key_size": 28, "raw_value_size": 108306012, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925420, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089381, "oldest_key_time": 0, "file_creation_time": 1732089381, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42531, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:24.247543 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042532.sst) /dev/shm/ToplingTemp/Topling-527435-lXhjno 2024/11/20-15:56:25.041797 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042532.sst): first pass time = 0.79's, 168.992'MB/sec 2024/11/20-15:56:26.179806 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042532.sst): old prealloc_size = 147639500, real_size = 33565392 2024/11/20-15:56:26.180339 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042532.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921375 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.67 avg-zval = 34.26 2024/11/20-15:56:26.181124 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042532.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:27.005553 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042532.sst): second pass time = 1.14's, 95.336'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 45.739'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3530.861'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.12's, 309.296'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 34623.387'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.336'MB/sec zip pipeline throughput = 95.336'MB/sec entries = 921375 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.67 avg-zval = 34.26 usrkeys = 921375 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.67 avg-zval = 34.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921375 bytes = 3340000 } seq expand size = 6449625 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4459 value = 3.4342 dict = 1.00 all = 3.5537 } Zip/UnZip{ index = 0.0957 value = 0.2912 dict = 1.00 all = 0.2814 } ---------------------------- total value len = 1.033522 GB avg = 0.124 KB (by entry num) total key len = 0.232585 GB avg = 0.028 KB total ukey len = 0.166132 GB avg = 0.020 KB total ukey num = 0.008306599 Billion total entry num = 0.008306599 Billion write speed all = 0.049216647 MB/sec (with seq num) write speed all = 0.046633464 MB/sec (without seq num) 2024/11/20-15:56:27.051674 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42532: 921375 keys, 37768208 bytes, temperature: kWarm 2024/11/20-15:56:27.051729 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089387051701, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42532, "file_size": 37768208, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 434681884, "largest_seqno": 738248369, "table_properties": {"data_size": 31570752, "index_size": 1764096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25798500, "raw_average_key_size": 28, "raw_value_size": 108419310, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921375, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089384, "oldest_key_time": 0, "file_creation_time": 1732089384, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42532, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:27.093350 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042533.sst) /dev/shm/ToplingTemp/Topling-527435-5jdK4j 2024/11/20-15:56:27.899086 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042533.sst): first pass time = 0.81's, 166.584'MB/sec 2024/11/20-15:56:29.033975 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042533.sst): old prealloc_size = 147639500, real_size = 35199920 2024/11/20-15:56:29.034552 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042533.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925791 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 116.98 avg-zval = 35.87 2024/11/20-15:56:29.035338 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042533.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:29.861860 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042533.sst): second pass time = 1.13's, 95.494'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.326'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3151.187'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.12's, 325.866'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25629.867'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 95.494'MB/sec zip pipeline throughput = 95.494'MB/sec entries = 925791 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.98 avg-zval = 35.87 usrkeys = 925791 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.98 avg-zval = 35.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925791 bytes = 3124560 } seq expand size = 6480537 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5139 value = 3.2612 dict = 1.00 all = 3.4257 } Zip/UnZip{ index = 0.0951 value = 0.3066 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 1.148298 GB avg = 0.124 KB (by entry num) total key len = 0.258507 GB avg = 0.028 KB total ukey len = 0.184648 GB avg = 0.020 KB total ukey num = 0.009232390 Billion total entry num = 0.009232390 Billion write speed all = 0.054679862 MB/sec (with seq num) write speed all = 0.051809097 MB/sec (without seq num) 2024/11/20-15:56:29.909425 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42533: 925791 keys, 39179118 bytes, temperature: kWarm 2024/11/20-15:56:29.909489 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089389909458, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42533, "file_size": 39179118, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 635922618, "largest_seqno": 738253301, "table_properties": {"data_size": 33207200, "index_size": 1761088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25922148, "raw_average_key_size": 28, "raw_value_size": 108295638, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925791, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089387, "oldest_key_time": 0, "file_creation_time": 1732089387, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42533, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:29.940443 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042534.sst) /dev/shm/ToplingTemp/Topling-527435-RMIjSn 2024/11/20-15:56:30.753238 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042534.sst): first pass time = 0.81's, 165.138'MB/sec 2024/11/20-15:56:31.875546 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042534.sst): old prealloc_size = 147639500, real_size = 35016672 2024/11/20-15:56:31.876115 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042534.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923650 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0330 GB avg-val = 117.31 avg-zval = 35.75 2024/11/20-15:56:31.876857 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042534.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:32.704333 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042534.sst): second pass time = 1.12's, 96.614'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.187'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3236.877'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.12's, 329.584'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32979.164'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 96.614'MB/sec zip pipeline throughput = 96.614'MB/sec entries = 923650 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.31 avg-zval = 35.75 usrkeys = 923650 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.31 avg-zval = 35.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923650 bytes = 3232784 } seq expand size = 6465550 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.4410 value = 3.2819 dict = 1.00 all = 3.4321 } Zip/UnZip{ index = 0.0958 value = 0.3047 dict = 1.00 all = 0.2914 } ---------------------------- total value len = 1.263119 GB avg = 0.124 KB (by entry num) total key len = 0.284369 GB avg = 0.028 KB total ukey len = 0.203121 GB avg = 0.020 KB total ukey num = 0.010156040 Billion total entry num = 0.010156040 Billion write speed all = 0.060141318 MB/sec (with seq num) write speed all = 0.056983697 MB/sec (without seq num) 2024/11/20-15:56:32.754286 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42534: 923650 keys, 39106976 bytes, temperature: kWarm 2024/11/20-15:56:32.754340 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089392754311, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42534, "file_size": 39106976, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 542657474, "largest_seqno": 738252004, "table_properties": {"data_size": 33016272, "index_size": 1769280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25862200, "raw_average_key_size": 28, "raw_value_size": 108355648, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923650, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089389, "oldest_key_time": 0, "file_creation_time": 1732089389, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42534, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:32.790778 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042535.sst) /dev/shm/ToplingTemp/Topling-527435-fSDCpo 2024/11/20-15:56:33.607444 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042535.sst): first pass time = 0.82's, 164.354'MB/sec 2024/11/20-15:56:34.756726 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042535.sst): old prealloc_size = 147639500, real_size = 37053520 2024/11/20-15:56:34.757310 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042535.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927713 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1082 GB zip-val = 0.0350 GB avg-val = 116.68 avg-zval = 37.75 2024/11/20-15:56:34.758033 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042535.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-15:56:35.528491 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042535.sst): second pass time = 1.15's, 94.252'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 53.110'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.664'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.359'MB/sec write SST data time = 0.12's, 329.409'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 17825.359'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 94.252'MB/sec zip pipeline throughput = 94.252'MB/sec entries = 927713 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.68 avg-zval = 37.75 usrkeys = 927713 avg-key = 20.00 avg-zkey = 1.94 avg-val = 122.78 avg-zval = 37.75 TagRS{ kinds = RS_Key0_TagN bytes = 145040 } TagArray{ size = 808751 bytes = 2931744 } seq expand size = 5661257 multi value expand size = 0 cnt WriteAppend = 35 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0350 GB dict = 1.08 MB all = 0.0410 GB } UnZip/Zip{ index = 10.3065 value = 3.0908 dict = 1.00 all = 3.2746 } Zip/UnZip{ index = 0.0970 value = 0.3235 dict = 1.00 all = 0.3054 } ---------------------------- total value len = 1.377022 GB avg = 0.124 KB (by entry num) total key len = 0.310345 GB avg = 0.028 KB total ukey len = 0.221675 GB avg = 0.020 KB total ukey num = 0.011083753 Billion total entry num = 0.011083753 Billion write speed all = 0.065570359 MB/sec (with seq num) write speed all = 0.062124680 MB/sec (without seq num) 2024/11/20-15:56:35.577518 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42535: 927713 keys, 40987948 bytes, temperature: kWarm 2024/11/20-15:56:35.577577 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089395577547, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42535, "file_size": 40987948, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 738248937, "table_properties": {"data_size": 35021120, "index_size": 1800256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975964, "raw_average_key_size": 28, "raw_value_size": 108241886, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927713, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089392, "oldest_key_time": 0, "file_creation_time": 1732089392, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42535, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:35.612741 140257533691456 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042536.sst) /dev/shm/ToplingTemp/Topling-527435-fDWddm 2024/11/20-15:56:35.683847 140257533691456 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042536.sst): first pass time = 0.07's, 254.754'MB/sec 2024/11/20-15:56:35.837955 140257533691456 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042536.sst): old prealloc_size = 147639500, real_size = 5392416 2024/11/20-15:56:35.838096 140257533691456 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042536.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 124023 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0146 GB zip-val = 0.0051 GB avg-val = 118.00 avg-zval = 41.30 2024/11/20-15:56:35.838229 140257533691456 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042536.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/20-15:56:35.946982 140257533691456 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00/042536.sst): second pass time = 0.15's, 95.350'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.11's, 54.300'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1978.201'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.336'MB/sec write SST data time = 0.02's, 357.083'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.154'MB, throughput = 5380.744'MB/sec zip my value time = 0.15's, unzip length = 0.015'GB zip my value throughput = 95.350'MB/sec zip pipeline throughput = 95.350'MB/sec entries = 124023 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 41.30 usrkeys = 124023 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 41.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 124023 bytes = 403088 } seq expand size = 868161 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0025 GB value = 0.0146 GB dict = 0.15 MB all = 0.0181 GB } __ZipSize{ index = 0.0002 GB value = 0.0051 GB dict = 0.15 MB all = 0.0059 GB } UnZip/Zip{ index = 10.3851 value = 2.8570 dict = 1.00 all = 3.0590 } Zip/UnZip{ index = 0.0963 value = 0.3500 dict = 1.00 all = 0.3269 } ---------------------------- total value len = 1.392525 GB avg = 0.124 KB (by entry num) total key len = 0.313818 GB avg = 0.028 KB total ukey len = 0.224156 GB avg = 0.020 KB total ukey num = 0.011207776 Billion total entry num = 0.011207776 Billion write speed all = 0.066306661 MB/sec (with seq num) write speed all = 0.062822483 MB/sec (without seq num) 2024/11/20-15:56:35.959793 140257533691456 [compaction_job.cc:1948] [default] [JOB 14213] Generated table #42536: 124023 keys, 5919437 bytes, temperature: kWarm 2024/11/20-15:56:35.959850 140257533691456 EVENT_LOG_v1 {"time_micros": 1732089395959816, "cf_name": "default", "job": 14213, "event": "table_file_creation", "file_number": 42536, "file_size": 5919437, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 671158717, "largest_seqno": 721170663, "table_properties": {"data_size": 5122384, "index_size": 238848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3472644, "raw_average_key_size": 28, "raw_value_size": 14634714, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 124023, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732089395, "oldest_key_time": 0, "file_creation_time": 1732089395, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 42536, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-15:56:35.971162 140257533691456 [compaction_job.cc:703] job-14213: subcompact[0], size: 0.478031 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14213/att-00 [042524,042525,042526,042527,042528,042529,042530,042531,042532,042533,042534,042535,042536] 2024/11/20-15:56:36.050222 140257533691456 (Original Log Time 2024/11/20-15:56:01.337795) EVENT_LOG_v1 {"time_micros": 1732089361337750, "job": 14213, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [42518], "files_L3": [42350, 42351, 42352, 42353, 42498, 42499, 42500, 42501, 42502, 42503, 42504, 42505], "score": 0.991698, "input_data_size": 500896968, "oldest_snapshot_seqno": 739349749} 2024/11/20-15:56:36.050225 140257533691456 (Original Log Time 2024/11/20-15:56:36.049056) [compaction_job.cc:2022] [default] [JOB 14213] Compacted 1@2 + 12@3 files to L3 => 478030543 bytes 2024/11/20-15:56:36.050227 140257533691456 (Original Log Time 2024/11/20-15:56:36.050155) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 14.5 rd, 13.8 wr, level 3, files in(1, 12) out(13 +0 blob) MB in(27.1, 450.6 +0.0 blob) out(455.9 +0.0 blob), read-write-amplify(34.5) write-amplify(16.8) OK, records in: 11316055, records dropped: 108279 output_compression: Snappy 2024/11/20-15:56:36.050231 140257533691456 (Original Log Time 2024/11/20-15:56:36.050204) EVENT_LOG_v1 {"time_micros": 1732089396050177, "cf": "default", "job": 14213, "event": "compaction_finished", "compaction_time_micros": 34633370, "compaction_time_cpu_micros": 16017257, "output_level": 3, "num_output_files": 13, "total_output_size": 478030543, "num_input_records": 11316055, "num_output_records": 11207776, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 305969898, "file_range_sync_nanos": 0, "file_fsync_nanos": 10669, "file_prepare_write_nanos": 29444, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/20-15:56:36.170349 140257533691456 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732076400,"starts":1732089361,"executesMs":34857,"compactionJobId":14213,"attempt":0,"compactionInputRawBytes":1644096718,"compactionInputZipBytes":500896968,"compactionOutputRawBytes":0,"compactionOutputZipBytes":478030543,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-e2009c5606e87e29e4d4229612b5e39e-85af63badfda12b5-00"}}, response = 2024/11/20-15:56:36.170400 140257533691456 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-14213/att-00: olev 3, work 34.857 s, result 7.686 ms, install 1.194 ms, input{raw 1.531 GiB zip 477.693 MiB}