2024/11/20-16:11:27.138077 140257751901760 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00: fork to child time = 0.010052 sec 2024/11/20-16:11:27.138596 140257751901760 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00 2024/11/20-16:11:27.145215 140257751901760 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00 2024/11/20-16:11:27.146819 140257751901760 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-16:11:27.147065 140257751901760 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-16:11:27.192969 140257751901760 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-14993/att-00/MANIFEST-019309 2024/11/20-16:11:27.243235 140257751901760 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-14993/att-00/MANIFEST-019309 succeeded,manifest_file_number is 19309, next_file_number is 43948, last_sequence is 756908736, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 43729 2024/11/20-16:11:27.243247 140257751901760 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-16:11:27.243330 140257751901760 [version_set.cc:5724] Creating manifest 43948 2024/11/20-16:11:27.276974 140257751901760 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-16:11:27.297387 140257751901760 [compaction_job.cc:2437] [default] [JOB 14993] Compacting 1@2 + 10@3 files to L3, score 0.94, subcompactions 1 : 1 2024/11/20-16:11:27.297399 140257751901760 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 11750 Base level 2, inputs: [43946(27MB)], [43840(36MB) 43841(38MB) 43842(36MB) 43843(37MB) 43844(37MB) 43845(37MB) 43846(37MB) 43847(39MB) 43848(37MB) 43927(38MB)] 2024/11/20-16:11:27.304377 140257751901760 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-16:11:27.304401 140257751901760 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-16:11:27.304421 140257751901760 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-16:11:27.304429 140257751901760 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-16:11:27.304439 140257751901760 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-16:11:27.844289 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043949.sst) /dev/shm/ToplingTemp/Topling-551303-IXqWR7 2024/11/20-16:11:28.432881 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043949.sst): first pass time = 0.59's, 228.053'MB/sec 2024/11/20-16:11:30.081402 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043949.sst): old prealloc_size = 147639500, real_size = 34014832 2024/11/20-16:11:30.082158 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043949.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920893 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.75 avg-zval = 34.75 2024/11/20-16:11:30.083127 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043949.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:11:31.525438 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043949.sst): second pass time = 1.65's, 65.816'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.44's, 26.385'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2452.884'MB/sec (index lex order gen) rebuild zvType time = 1.21's, 0.190'MB/sec write SST data time = 0.23's, 165.819'MB/sec dict compress time = 3687948028.52's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 25072.796'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.816'MB/sec zip pipeline throughput = 145.711'MB/sec entries = 920893 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.75 avg-zval = 34.75 usrkeys = 920893 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.75 avg-zval = 34.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920893 bytes = 3223136 } seq expand size = 6446251 multi value expand size = 0 cnt WriteAppend = 27 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.3087 value = 3.3888 dict = 1.00 all = 3.5226 } Zip/UnZip{ index = 0.0970 value = 0.2951 dict = 1.00 all = 0.2839 } ---------------------------- total value len = 0.114879 GB avg = 0.125 KB (by entry num) total key len = 0.025785 GB avg = 0.028 KB total ukey len = 0.018418 GB avg = 0.020 KB total ukey num = 0.000920893 Billion total entry num = 0.000920893 Billion write speed all = 0.005282229 MB/sec (with seq num) write speed all = 0.005005578 MB/sec (without seq num) 2024/11/20-16:11:31.578309 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43949: 920893 keys, 38101480 bytes, temperature: kWarm 2024/11/20-16:11:31.578383 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090291578345, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43949, "file_size": 38101480, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 522320272, "largest_seqno": 756105188, "table_properties": {"data_size": 31997792, "index_size": 1786624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25785004, "raw_average_key_size": 28, "raw_value_size": 108432862, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920893, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090287, "oldest_key_time": 0, "file_creation_time": 1732090287, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43949, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:11:31.624380 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043950.sst) /dev/shm/ToplingTemp/Topling-551303-9Ipj77 2024/11/20-16:11:32.607289 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043950.sst): first pass time = 0.98's, 136.559'MB/sec 2024/11/20-16:11:33.892292 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043950.sst): old prealloc_size = 147639500, real_size = 37035840 2024/11/20-16:11:33.893475 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043950.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928714 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0350 GB avg-val = 116.52 avg-zval = 37.71 2024/11/20-16:11:33.894395 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043950.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:11:34.858414 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043950.sst): second pass time = 1.28's, 84.271'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 42.584'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1535.987'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.299'MB/sec write SST data time = 0.19's, 217.343'MB/sec dict compress time = -7404648962.61's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18057.357'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.271'MB/sec zip pipeline throughput = 84.271'MB/sec entries = 928714 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.52 avg-zval = 37.71 usrkeys = 928714 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.52 avg-zval = 37.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928714 bytes = 3250512 } seq expand size = 6500998 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0350 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.4449 value = 3.0896 dict = 1.00 all = 3.2625 } Zip/UnZip{ index = 0.0957 value = 0.3237 dict = 1.00 all = 0.3065 } ---------------------------- total value len = 0.229594 GB avg = 0.124 KB (by entry num) total key len = 0.051789 GB avg = 0.028 KB total ukey len = 0.036992 GB avg = 0.020 KB total ukey num = 0.001849607 Billion total entry num = 0.001849607 Billion write speed all = 0.010565193 MB/sec (with seq num) write speed all = 0.010009610 MB/sec (without seq num) 2024/11/20-16:11:34.914593 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43950: 928714 keys, 41139896 bytes, temperature: kWarm 2024/11/20-16:11:34.914682 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090294914637, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43950, "file_size": 41139896, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 538603295, "largest_seqno": 756104910, "table_properties": {"data_size": 35025200, "index_size": 1778304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003992, "raw_average_key_size": 28, "raw_value_size": 108213880, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928714, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090291, "oldest_key_time": 0, "file_creation_time": 1732090291, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43950, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:11:34.959924 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043951.sst) /dev/shm/ToplingTemp/Topling-551303-WrebN9 2024/11/20-16:11:36.261394 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043951.sst): first pass time = 1.30's, 103.131'MB/sec 2024/11/20-16:11:37.858879 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043951.sst): old prealloc_size = 147639500, real_size = 34930752 2024/11/20-16:11:37.859678 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043951.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922424 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0329 GB avg-val = 117.51 avg-zval = 35.68 2024/11/20-16:11:37.860618 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043951.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:11:38.834942 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043951.sst): second pass time = 1.60's, 67.885'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 39.978'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2321.101'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.17's, 234.718'MB/sec dict compress time = -7404648962.61's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 28136.915'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.885'MB/sec zip pipeline throughput = 86.654'MB/sec entries = 922424 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.51 avg-zval = 35.68 usrkeys = 922424 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.51 avg-zval = 35.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922424 bytes = 3228496 } seq expand size = 6456968 multi value expand size = 0 cnt WriteAppend = 57 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.3074 value = 3.2935 dict = 1.00 all = 3.4396 } Zip/UnZip{ index = 0.0970 value = 0.3036 dict = 1.00 all = 0.2907 } ---------------------------- total value len = 0.344441 GB avg = 0.124 KB (by entry num) total key len = 0.077617 GB avg = 0.028 KB total ukey len = 0.055441 GB avg = 0.020 KB total ukey num = 0.002772031 Billion total entry num = 0.002772031 Billion write speed all = 0.015844793 MB/sec (with seq num) write speed all = 0.015012257 MB/sec (without seq num) 2024/11/20-16:11:38.885675 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43951: 922424 keys, 39021146 bytes, temperature: kWarm 2024/11/20-16:11:38.885728 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090298885701, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43951, "file_size": 39021146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 503040251, "largest_seqno": 756103730, "table_properties": {"data_size": 32910128, "index_size": 1789824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25827872, "raw_average_key_size": 28, "raw_value_size": 108389868, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090294, "oldest_key_time": 0, "file_creation_time": 1732090294, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43951, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:11:38.946381 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043952.sst) /dev/shm/ToplingTemp/Topling-551303-bnrYJ7 2024/11/20-16:11:39.832449 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043952.sst): first pass time = 0.89's, 151.483'MB/sec 2024/11/20-16:11:41.229820 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043952.sst): old prealloc_size = 147639500, real_size = 35243536 2024/11/20-16:11:41.230490 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043952.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923815 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.29 avg-zval = 36.00 2024/11/20-16:11:41.231314 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043952.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:11:42.291425 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043952.sst): second pass time = 1.40's, 77.578'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 37.052'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2717.542'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.256'MB/sec write SST data time = 0.16's, 249.858'MB/sec dict compress time = -7404648962.61's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 28720.493'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.578'MB/sec zip pipeline throughput = 77.578'MB/sec entries = 923815 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.29 avg-zval = 36.00 usrkeys = 923815 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.29 avg-zval = 36.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923815 bytes = 3233360 } seq expand size = 6466705 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5140 value = 3.2582 dict = 1.00 all = 3.4123 } Zip/UnZip{ index = 0.0951 value = 0.3069 dict = 1.00 all = 0.2931 } ---------------------------- total value len = 0.459259 GB avg = 0.124 KB (by entry num) total key len = 0.103484 GB avg = 0.028 KB total ukey len = 0.073917 GB avg = 0.020 KB total ukey num = 0.003695846 Billion total entry num = 0.003695846 Billion write speed all = 0.021123599 MB/sec (with seq num) write speed all = 0.020013754 MB/sec (without seq num) 2024/11/20-16:11:42.343081 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43952: 923815 keys, 39333894 bytes, temperature: kWarm 2024/11/20-16:11:42.343142 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090302343112, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43952, "file_size": 39333894, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 591115717, "largest_seqno": 756104732, "table_properties": {"data_size": 33255104, "index_size": 1757312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25866820, "raw_average_key_size": 28, "raw_value_size": 108351014, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923815, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090298, "oldest_key_time": 0, "file_creation_time": 1732090298, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43952, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:11:42.384601 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043953.sst) /dev/shm/ToplingTemp/Topling-551303-Hvr2q7 2024/11/20-16:11:43.584906 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043953.sst): first pass time = 1.20's, 111.823'MB/sec 2024/11/20-16:11:44.860877 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043953.sst): old prealloc_size = 147639500, real_size = 35772928 2024/11/20-16:11:44.861574 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043953.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923853 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 117.28 avg-zval = 36.57 2024/11/20-16:11:44.862360 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043953.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:11:45.764143 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043953.sst): second pass time = 1.28's, 84.966'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 44.006'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2621.756'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.14's, 293.822'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 25787.132'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.966'MB/sec zip pipeline throughput = 84.966'MB/sec entries = 923853 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 36.57 usrkeys = 923853 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.28 avg-zval = 36.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923853 bytes = 3118016 } seq expand size = 6466971 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5140 value = 3.2071 dict = 1.00 all = 3.3767 } Zip/UnZip{ index = 0.0951 value = 0.3118 dict = 1.00 all = 0.2961 } ---------------------------- total value len = 0.574075 GB avg = 0.124 KB (by entry num) total key len = 0.129352 GB avg = 0.028 KB total ukey len = 0.092394 GB avg = 0.020 KB total ukey num = 0.004619699 Billion total entry num = 0.004619699 Billion write speed all = 0.026401029 MB/sec (with seq num) write speed all = 0.025013936 MB/sec (without seq num) 2024/11/20-16:11:45.812772 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43953: 923853 keys, 39748390 bytes, temperature: kWarm 2024/11/20-16:11:45.812840 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090305812809, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43953, "file_size": 39748390, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 625722464, "largest_seqno": 756104787, "table_properties": {"data_size": 33784432, "index_size": 1757376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25867884, "raw_average_key_size": 28, "raw_value_size": 108349958, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923853, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090302, "oldest_key_time": 0, "file_creation_time": 1732090302, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43953, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:11:45.864793 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043954.sst) /dev/shm/ToplingTemp/Topling-551303-swfUe8 2024/11/20-16:11:46.741416 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043954.sst): first pass time = 0.88's, 153.116'MB/sec 2024/11/20-16:11:48.408037 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043954.sst): old prealloc_size = 147639500, real_size = 34995968 2024/11/20-16:11:48.408653 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043954.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922319 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0330 GB avg-val = 117.52 avg-zval = 35.79 2024/11/20-16:11:48.409436 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043954.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:11:49.408121 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043954.sst): second pass time = 1.67's, 65.071'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 39.080'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3030.231'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.267'MB/sec write SST data time = 0.14's, 287.274'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 19141.207'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 65.071'MB/sec zip pipeline throughput = 197.351'MB/sec entries = 922319 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 35.79 usrkeys = 922319 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.52 avg-zval = 35.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922319 bytes = 3228128 } seq expand size = 6456233 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5119 value = 3.2836 dict = 1.00 all = 3.4342 } Zip/UnZip{ index = 0.0951 value = 0.3045 dict = 1.00 all = 0.2912 } ---------------------------- total value len = 0.688925 GB avg = 0.124 KB (by entry num) total key len = 0.155177 GB avg = 0.028 KB total ukey len = 0.110840 GB avg = 0.020 KB total ukey num = 0.005542018 Billion total entry num = 0.005542018 Billion write speed all = 0.031676476 MB/sec (with seq num) write speed all = 0.030012680 MB/sec (without seq num) 2024/11/20-16:11:49.455503 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43954: 922319 keys, 39082998 bytes, temperature: kWarm 2024/11/20-16:11:49.455564 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090309455535, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43954, "file_size": 39082998, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 591115808, "largest_seqno": 756104823, "table_properties": {"data_size": 33010416, "index_size": 1754816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25824932, "raw_average_key_size": 28, "raw_value_size": 108392898, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922319, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090305, "oldest_key_time": 0, "file_creation_time": 1732090305, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43954, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:11:49.491887 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043955.sst) /dev/shm/ToplingTemp/Topling-551303-msBUd9 2024/11/20-16:11:50.328970 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043955.sst): first pass time = 0.84's, 160.347'MB/sec 2024/11/20-16:11:51.594799 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043955.sst): old prealloc_size = 147639500, real_size = 34784592 2024/11/20-16:11:51.595458 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043955.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923460 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0328 GB avg-val = 117.34 avg-zval = 35.48 2024/11/20-16:11:51.596283 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043955.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:11:52.548407 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043955.sst): second pass time = 1.27's, 85.653'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 40.891'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2819.608'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 291.881'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33434.632'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.653'MB/sec zip pipeline throughput = 85.653'MB/sec entries = 923460 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.34 avg-zval = 35.48 usrkeys = 923460 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.34 avg-zval = 35.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923460 bytes = 3347552 } seq expand size = 6464220 multi value expand size = 0 cnt WriteAppend = 37 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.3061 value = 3.3076 dict = 1.00 all = 3.4420 } Zip/UnZip{ index = 0.0970 value = 0.3023 dict = 1.00 all = 0.2905 } ---------------------------- total value len = 0.803750 GB avg = 0.124 KB (by entry num) total key len = 0.181033 GB avg = 0.028 KB total ukey len = 0.129310 GB avg = 0.020 KB total ukey num = 0.006465478 Billion total entry num = 0.006465478 Billion write speed all = 0.036951477 MB/sec (with seq num) write speed all = 0.035010672 MB/sec (without seq num) 2024/11/20-16:11:52.600890 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43955: 923460 keys, 38993614 bytes, temperature: kWarm 2024/11/20-16:11:52.600941 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090312600912, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43955, "file_size": 38993614, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 431720469, "largest_seqno": 756105667, "table_properties": {"data_size": 32761472, "index_size": 1792064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25856880, "raw_average_key_size": 28, "raw_value_size": 108360884, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923460, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090309, "oldest_key_time": 0, "file_creation_time": 1732090309, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43955, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:11:52.636380 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043956.sst) /dev/shm/ToplingTemp/Topling-551303-jxUFW9 2024/11/20-16:11:53.464110 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043956.sst): first pass time = 0.83's, 162.157'MB/sec 2024/11/20-16:11:54.731910 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043956.sst): old prealloc_size = 147639500, real_size = 37027856 2024/11/20-16:11:54.732534 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043956.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926414 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0350 GB avg-val = 116.88 avg-zval = 37.80 2024/11/20-16:11:54.733351 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043956.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:11:55.541655 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043956.sst): second pass time = 1.27's, 85.455'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 50.936'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2949.629'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.344'MB/sec write SST data time = 0.14's, 304.978'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33920.372'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.455'MB/sec zip pipeline throughput = 85.455'MB/sec entries = 926414 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.88 avg-zval = 37.80 usrkeys = 926414 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.88 avg-zval = 37.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926414 bytes = 3358272 } seq expand size = 6484898 multi value expand size = 0 cnt WriteAppend = 37 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0350 GB dict = 1.08 MB all = 0.0412 GB } UnZip/Zip{ index = 10.4443 value = 3.0917 dict = 1.00 all = 3.2542 } Zip/UnZip{ index = 0.0957 value = 0.3234 dict = 1.00 all = 0.3073 } ---------------------------- total value len = 0.918513 GB avg = 0.124 KB (by entry num) total key len = 0.206973 GB avg = 0.028 KB total ukey len = 0.147838 GB avg = 0.020 KB total ukey num = 0.007391892 Billion total entry num = 0.007391892 Billion write speed all = 0.042226247 MB/sec (with seq num) write speed all = 0.040007601 MB/sec (without seq num) 2024/11/20-16:11:55.600084 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43956: 926414 keys, 41244472 bytes, temperature: kWarm 2024/11/20-16:11:55.600150 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090315600119, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43956, "file_size": 41244472, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 431942051, "largest_seqno": 756105825, "table_properties": {"data_size": 35022080, "index_size": 1774016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939592, "raw_average_key_size": 28, "raw_value_size": 108278268, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926414, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090312, "oldest_key_time": 0, "file_creation_time": 1732090312, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43956, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:11:55.639142 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043957.sst) /dev/shm/ToplingTemp/Topling-551303-DGhKp8 2024/11/20-16:11:56.704474 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043957.sst): first pass time = 1.07's, 125.991'MB/sec 2024/11/20-16:11:57.868421 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043957.sst): old prealloc_size = 147639500, real_size = 34623008 2024/11/20-16:11:57.868979 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043957.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923924 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 117.27 avg-zval = 35.32 2024/11/20-16:11:57.869802 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043957.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:11:58.718940 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043957.sst): second pass time = 1.16's, 93.145'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.519'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3291.849'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 302.526'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 19320.278'MB/sec zip my value time = 1.16's, unzip length = 0.108'GB zip my value throughput = 93.145'MB/sec zip pipeline throughput = 93.145'MB/sec entries = 923924 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.27 avg-zval = 35.32 usrkeys = 923924 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.27 avg-zval = 35.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923924 bytes = 3233744 } seq expand size = 6467468 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5095 value = 3.3201 dict = 1.00 all = 3.4669 } Zip/UnZip{ index = 0.0952 value = 0.3012 dict = 1.00 all = 0.2884 } ---------------------------- total value len = 1.033328 GB avg = 0.124 KB (by entry num) total key len = 0.232843 GB avg = 0.028 KB total ukey len = 0.166316 GB avg = 0.020 KB total ukey num = 0.008315816 Billion total entry num = 0.008315816 Billion write speed all = 0.047498846 MB/sec (with seq num) write speed all = 0.045003186 MB/sec (without seq num) 2024/11/20-16:11:58.774665 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43957: 923924 keys, 38713890 bytes, temperature: kWarm 2024/11/20-16:11:58.774731 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090318774700, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43957, "file_size": 38713890, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 542657474, "largest_seqno": 756105631, "table_properties": {"data_size": 32633552, "index_size": 1758272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869872, "raw_average_key_size": 28, "raw_value_size": 108347876, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923924, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090315, "oldest_key_time": 0, "file_creation_time": 1732090315, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43957, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:11:58.826359 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043958.sst) /dev/shm/ToplingTemp/Topling-551303-nag659 2024/11/20-16:11:59.759905 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043958.sst): first pass time = 0.93's, 143.779'MB/sec 2024/11/20-16:12:01.239228 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043958.sst): old prealloc_size = 147639500, real_size = 36802320 2024/11/20-16:12:01.240014 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043958.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927714 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1082 GB zip-val = 0.0348 GB avg-val = 116.68 avg-zval = 37.48 2024/11/20-16:12:01.240917 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043958.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-16:12:02.179278 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043958.sst): second pass time = 1.48's, 73.218'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 43.335'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2396.365'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.17's, 244.557'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32088.160'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.218'MB/sec zip pipeline throughput = 112.200'MB/sec entries = 927714 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.68 avg-zval = 37.48 usrkeys = 927714 avg-key = 20.00 avg-zkey = 1.94 avg-val = 122.78 avg-zval = 37.48 TagRS{ kinds = RS_Key0_TagN bytes = 145040 } TagArray{ size = 808752 bytes = 2931744 } seq expand size = 5661264 multi value expand size = 0 cnt WriteAppend = 35 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0407 GB } UnZip/Zip{ index = 10.3072 value = 3.1131 dict = 1.00 all = 3.2947 } Zip/UnZip{ index = 0.0970 value = 0.3212 dict = 1.00 all = 0.3035 } ---------------------------- total value len = 1.147231 GB avg = 0.124 KB (by entry num) total key len = 0.258819 GB avg = 0.028 KB total ukey len = 0.184871 GB avg = 0.020 KB total ukey num = 0.009243530 Billion total entry num = 0.009243530 Billion write speed all = 0.052739393 MB/sec (with seq num) write speed all = 0.049965676 MB/sec (without seq num) 2024/11/20-16:12:02.258675 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43958: 927714 keys, 40737064 bytes, temperature: kWarm 2024/11/20-16:12:02.258737 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090322258707, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43958, "file_size": 40737064, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 756104548, "table_properties": {"data_size": 34770048, "index_size": 1800128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975992, "raw_average_key_size": 28, "raw_value_size": 108241872, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927714, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090318, "oldest_key_time": 0, "file_creation_time": 1732090318, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43958, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:12:02.295226 140257751901760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043959.sst) /dev/shm/ToplingTemp/Topling-551303-9cqXe8 2024/11/20-16:12:02.386351 140257751901760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043959.sst): first pass time = 0.09's, 250.892'MB/sec 2024/11/20-16:12:02.575759 140257751901760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043959.sst): old prealloc_size = 147639500, real_size = 6649440 2024/11/20-16:12:02.575909 140257751901760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043959.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 156532 prefix = 4 raw-key = 0.0025 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0185 GB zip-val = 0.0063 GB avg-val = 118.00 avg-zval = 40.31 2024/11/20-16:12:02.576106 140257751901760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043959.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.023 GB 2024/11/20-16:12:02.716540 140257751901760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00/043959.sst): second pass time = 0.19's, 97.921'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 51.949'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2450.753'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.333'MB/sec write SST data time = 0.02's, 317.847'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.193'MB, throughput = 3751.969'MB/sec zip my value time = 0.19's, unzip length = 0.018'GB zip my value throughput = 97.921'MB/sec zip pipeline throughput = 97.921'MB/sec entries = 156532 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 40.31 usrkeys = 156532 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 40.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 156532 bytes = 508736 } seq expand size = 1095724 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0031 GB value = 0.0185 GB dict = 0.19 MB all = 0.0229 GB } __ZipSize{ index = 0.0003 GB value = 0.0063 GB dict = 0.19 MB all = 0.0073 GB } UnZip/Zip{ index = 10.4033 value = 2.9276 dict = 1.00 all = 3.1249 } Zip/UnZip{ index = 0.0961 value = 0.3416 dict = 1.00 all = 0.3200 } ---------------------------- total value len = 1.166798 GB avg = 0.124 KB (by entry num) total key len = 0.263202 GB avg = 0.028 KB total ukey len = 0.188001 GB avg = 0.020 KB total ukey num = 0.009400062 Billion total entry num = 0.009400062 Billion write speed all = 0.053636628 MB/sec (with seq num) write speed all = 0.050815997 MB/sec (without seq num) 2024/11/20-16:12:02.733698 140257751901760 [compaction_job.cc:1948] [default] [JOB 14993] Generated table #43959: 156532 keys, 7313473 bytes, temperature: kWarm 2024/11/20-16:12:02.733762 140257751901760 EVENT_LOG_v1 {"time_micros": 1732090322733733, "cf_name": "default", "job": 14993, "event": "table_file_creation", "file_number": 43959, "file_size": 7313473, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 665727317, "largest_seqno": 728925397, "table_properties": {"data_size": 6309200, "index_size": 300928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4382896, "raw_average_key_size": 28, "raw_value_size": 18470776, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 156532, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732090322, "oldest_key_time": 0, "file_creation_time": 1732090322, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 43959, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-16:12:02.746548 140257751901760 [compaction_job.cc:703] job-14993: subcompact[0], size: 0.403430 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14993/att-00 [043949,043950,043951,043952,043953,043954,043955,043956,043957,043958,043959] 2024/11/20-16:12:02.823700 140257751901760 (Original Log Time 2024/11/20-16:11:27.297454) EVENT_LOG_v1 {"time_micros": 1732090287297413, "job": 14993, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [43946], "files_L3": [43840, 43841, 43842, 43843, 43844, 43845, 43846, 43847, 43848, 43927], "score": 0.937986, "input_data_size": 424558816, "oldest_snapshot_seqno": 756908698} 2024/11/20-16:12:02.823704 140257751901760 (Original Log Time 2024/11/20-16:12:02.822698) [compaction_job.cc:2022] [default] [JOB 14993] Compacted 1@2 + 10@3 files to L3 => 403430317 bytes 2024/11/20-16:12:02.823705 140257751901760 (Original Log Time 2024/11/20-16:12:02.823630) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.02, MB/sec: 12.0 rd, 11.4 wr, level 3, files in(1, 10) out(11 +0 blob) MB in(27.1, 377.8 +0.0 blob) out(384.7 +0.0 blob), read-write-amplify(29.1) write-amplify(14.2) OK, records in: 9475478, records dropped: 75416 output_compression: Snappy 2024/11/20-16:12:02.823710 140257751901760 (Original Log Time 2024/11/20-16:12:02.823678) EVENT_LOG_v1 {"time_micros": 1732090322823653, "cf": "default", "job": 14993, "event": "compaction_finished", "compaction_time_micros": 35449105, "compaction_time_cpu_micros": 16254316, "output_level": 3, "num_output_files": 11, "total_output_size": 403430317, "num_input_records": 9475478, "num_output_records": 9400062, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 315464450, "file_range_sync_nanos": 0, "file_fsync_nanos": 10583, "file_prepare_write_nanos": 28819, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/20-16:12:02.964011 140257751901760 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732076400,"starts":1732090287,"executesMs":35678,"compactionJobId":14993,"attempt":0,"compactionInputRawBytes":1375740964,"compactionInputZipBytes":424558816,"compactionOutputRawBytes":0,"compactionOutputZipBytes":403430317,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-d6d5cf28de9bdade25d3b0eb536c45c2-a9d8dfb751dfb6af-00"}}, response = 2024/11/20-16:12:02.964112 140257751901760 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-14993/att-00: olev 3, work 35.678 s, result 5.917 ms, install 1.027 ms, input{raw 1.281 GiB zip 404.891 MiB}