2024/11/20-21:10:07.577073 140008198919744 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00: fork to child time = 0.009780 sec 2024/11/20-21:10:07.577642 140008198919744 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00 2024/11/20-21:10:07.584360 140008198919744 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00 2024/11/20-21:10:07.585966 140008198919744 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-21:10:07.586209 140008198919744 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-21:10:07.632352 140008198919744 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-13258/att-00/MANIFEST-049091 2024/11/20-21:10:07.648412 140008198919744 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-13258/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 71382, last_sequence is 1140919920, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 71199 2024/11/20-21:10:07.648418 140008198919744 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-21:10:07.648511 140008198919744 [version_set.cc:5724] Creating manifest 71382 2024/11/20-21:10:07.681112 140008198919744 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-21:10:07.706656 140008198919744 [compaction_job.cc:2437] [default] [JOB 13258] Compacting 1@2 + 17@3 files to L3, score 0.91, subcompactions 1 : 1 2024/11/20-21:10:07.706680 140008198919744 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 10705 Base level 2, inputs: [71375(54MB)], [71326(39MB) 71327(36MB) 71328(37MB) 71329(36MB) 71330(37MB) 71331(36MB) 71332(36MB) 71333(39MB) 71334(38MB) 71335(38MB) 71336(38MB) 71337(38MB) 71338(38MB) 71271(36MB) 71272(37MB) 71273(37MB) 71274(38MB)] 2024/11/20-21:10:07.722318 140008198919744 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-21:10:07.722345 140008198919744 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-21:10:07.722372 140008198919744 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-21:10:07.722382 140008198919744 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-21:10:07.722394 140008198919744 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-21:10:08.587863 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071383.sst) /dev/shm/ToplingTemp/Topling-993471-R7592M 2024/11/20-21:10:09.357018 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071383.sst): first pass time = 0.77's, 174.520'MB/sec 2024/11/20-21:10:11.304522 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071383.sst): old prealloc_size = 147639500, real_size = 37798768 2024/11/20-21:10:11.305332 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071383.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922748 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0358 GB avg-val = 117.45 avg-zval = 38.77 2024/11/20-21:10:11.307019 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071383.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:12.488284 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071383.sst): second pass time = 1.95's, 55.681'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.18's, 35.387'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2282.378'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.227'MB/sec write SST data time = 0.17's, 250.797'MB/sec dict compress time = 5501251829.64's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 23487.941'MB/sec zip my value time = 1.95's, unzip length = 0.108'GB zip my value throughput = 55.681'MB/sec zip pipeline throughput = 176.753'MB/sec entries = 922748 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.45 avg-zval = 38.77 usrkeys = 922748 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.45 avg-zval = 38.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922748 bytes = 3229632 } seq expand size = 6459236 multi value expand size = 0 cnt WriteAppend = 37 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0358 GB dict = 1.08 MB all = 0.0419 GB } UnZip/Zip{ index = 10.3074 value = 3.0293 dict = 1.00 all = 3.2040 } Zip/UnZip{ index = 0.0970 value = 0.3301 dict = 1.00 all = 0.3121 } ---------------------------- total value len = 563.180572 GB avg = 0.331 KB (by entry num) total key len = 44.340243 GB avg = 0.026 KB total ukey len = 32.812597 GB avg = 0.014 KB total ukey num = 2.321937324 Billion total entry num = 1.701439346 Billion write speed all = 54.290726460 MB/sec (with seq num) write speed all = 53.074341760 MB/sec (without seq num) 2024/11/20-21:10:12.539148 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71383: 922748 keys, 41890230 bytes, temperature: kWarm 2024/11/20-21:10:12.539250 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108212539206, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71383, "file_size": 41890230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 939479839, "largest_seqno": 1140424537, "table_properties": {"data_size": 35777440, "index_size": 1790464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25836944, "raw_average_key_size": 28, "raw_value_size": 108380796, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922748, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108208, "oldest_key_time": 0, "file_creation_time": 1732108208, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71383, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:12.584628 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071384.sst) /dev/shm/ToplingTemp/Topling-993471-dK3ZaO 2024/11/20-21:10:13.503637 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071384.sst): first pass time = 0.92's, 146.053'MB/sec 2024/11/20-21:10:14.940974 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071384.sst): old prealloc_size = 147639500, real_size = 34447296 2024/11/20-21:10:14.941704 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071384.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923850 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 117.28 avg-zval = 35.13 2024/11/20-21:10:14.942619 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071384.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:15.948017 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071384.sst): second pass time = 1.44's, 75.434'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 38.269'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2489.294'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.273'MB/sec write SST data time = 0.16's, 241.850'MB/sec dict compress time = 1080864464.01's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35519.903'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.434'MB/sec zip pipeline throughput = 75.434'MB/sec entries = 923850 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 35.13 usrkeys = 923850 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.28 avg-zval = 35.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923850 bytes = 3233488 } seq expand size = 6466950 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5144 value = 3.3381 dict = 1.00 all = 3.4827 } Zip/UnZip{ index = 0.0951 value = 0.2996 dict = 1.00 all = 0.2871 } ---------------------------- total value len = 563.295389 GB avg = 0.331 KB (by entry num) total key len = 44.366111 GB avg = 0.026 KB total ukey len = 32.831074 GB avg = 0.014 KB total ukey num = 2.322861174 Billion total entry num = 1.702363196 Billion write speed all = 54.286514526 MB/sec (with seq num) write speed all = 53.069845518 MB/sec (without seq num) 2024/11/20-21:10:15.994220 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71384: 923850 keys, 38538094 bytes, temperature: kWarm 2024/11/20-21:10:15.994279 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108215994247, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71384, "file_size": 38538094, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 952414140, "largest_seqno": 1140425194, "table_properties": {"data_size": 32458864, "index_size": 1757312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25867800, "raw_average_key_size": 28, "raw_value_size": 108349956, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923850, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108212, "oldest_key_time": 0, "file_creation_time": 1732108212, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71384, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:16.031293 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071385.sst) /dev/shm/ToplingTemp/Topling-993471-ziKSgN 2024/11/20-21:10:16.973665 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071385.sst): first pass time = 0.94's, 142.431'MB/sec 2024/11/20-21:10:18.920531 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071385.sst): old prealloc_size = 147639500, real_size = 35743072 2024/11/20-21:10:18.921100 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071385.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927800 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 116.66 avg-zval = 36.37 2024/11/20-21:10:18.921876 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071385.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:19.898957 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071385.sst): second pass time = 1.95's, 55.621'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 40.605'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.127'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.304'MB/sec write SST data time = 0.22's, 184.018'MB/sec dict compress time = 1080864464.01's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 20468.413'MB/sec zip my value time = 1.95's, unzip length = 0.108'GB zip my value throughput = 55.621'MB/sec zip pipeline throughput = 186.673'MB/sec entries = 927800 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 36.37 usrkeys = 927800 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 36.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927800 bytes = 3131344 } seq expand size = 6494600 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5126 value = 3.2075 dict = 1.00 all = 3.3783 } Zip/UnZip{ index = 0.0951 value = 0.3118 dict = 1.00 all = 0.2960 } ---------------------------- total value len = 563.410123 GB avg = 0.331 KB (by entry num) total key len = 44.392089 GB avg = 0.026 KB total ukey len = 32.849630 GB avg = 0.014 KB total ukey num = 2.323788974 Billion total entry num = 1.703290996 Billion write speed all = 54.279926450 MB/sec (with seq num) write speed all = 53.063023872 MB/sec (without seq num) 2024/11/20-21:10:19.946426 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71385: 927800 keys, 39728820 bytes, temperature: kWarm 2024/11/20-21:10:19.946484 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108219946454, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71385, "file_size": 39728820, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1069697620, "largest_seqno": 1140424586, "table_properties": {"data_size": 33745808, "index_size": 1765120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25978400, "raw_average_key_size": 28, "raw_value_size": 108239332, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927800, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108216, "oldest_key_time": 0, "file_creation_time": 1732108216, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71385, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:19.983971 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071386.sst) /dev/shm/ToplingTemp/Topling-993471-oASFlQ 2024/11/20-21:10:20.969966 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071386.sst): first pass time = 0.99's, 136.129'MB/sec 2024/11/20-21:10:22.447856 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071386.sst): old prealloc_size = 147639500, real_size = 34207616 2024/11/20-21:10:22.448667 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071386.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923660 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0322 GB avg-val = 117.31 avg-zval = 34.88 2024/11/20-21:10:22.449516 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071386.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:23.527303 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071386.sst): second pass time = 1.48's, 73.360'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 35.480'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2233.057'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.257'MB/sec write SST data time = 0.18's, 213.406'MB/sec dict compress time = 1080864464.01's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 27655.759'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.360'MB/sec zip pipeline throughput = 73.360'MB/sec entries = 923660 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.31 avg-zval = 34.88 usrkeys = 923660 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.31 avg-zval = 34.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923660 bytes = 3232832 } seq expand size = 6465620 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5130 value = 3.3631 dict = 1.00 all = 3.5046 } Zip/UnZip{ index = 0.0951 value = 0.2973 dict = 1.00 all = 0.2853 } ---------------------------- total value len = 563.524944 GB avg = 0.331 KB (by entry num) total key len = 44.417952 GB avg = 0.026 KB total ukey len = 32.868103 GB avg = 0.014 KB total ukey num = 2.324712634 Billion total entry num = 1.704214656 Billion write speed all = 54.274903523 MB/sec (with seq num) write speed all = 53.057735442 MB/sec (without seq num) 2024/11/20-21:10:23.574534 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71386: 923660 keys, 38298016 bytes, temperature: kWarm 2024/11/20-21:10:23.574599 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108223574567, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71386, "file_size": 38298016, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 952414237, "largest_seqno": 1140425026, "table_properties": {"data_size": 32219312, "index_size": 1757184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25862480, "raw_average_key_size": 28, "raw_value_size": 108355316, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923660, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108219, "oldest_key_time": 0, "file_creation_time": 1732108219, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71386, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:23.613095 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071387.sst) /dev/shm/ToplingTemp/Topling-993471-yx4HLP 2024/11/20-21:10:24.861723 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071387.sst): first pass time = 1.25's, 107.496'MB/sec 2024/11/20-21:10:26.357727 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071387.sst): old prealloc_size = 147639500, real_size = 34666672 2024/11/20-21:10:26.358295 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071387.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924155 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 117.23 avg-zval = 35.30 2024/11/20-21:10:26.359102 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071387.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:27.337734 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071387.sst): second pass time = 1.50's, 72.459'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 39.521'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3331.476'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.277'MB/sec write SST data time = 0.15's, 264.072'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 14595.556'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.459'MB/sec zip pipeline throughput = 160.493'MB/sec entries = 924155 avg-key = 28.00 avg-zkey = 1.97 avg-val = 117.23 avg-zval = 35.30 usrkeys = 924155 avg-key = 20.00 avg-zkey = 1.97 avg-val = 123.62 avg-zval = 35.30 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 843663 bytes = 3058288 } seq expand size = 5905641 multi value expand size = 0 cnt WriteAppend = 89 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.1758 value = 3.3214 dict = 1.00 all = 3.4655 } Zip/UnZip{ index = 0.0983 value = 0.3011 dict = 1.00 all = 0.2886 } ---------------------------- total value len = 563.639191 GB avg = 0.331 KB (by entry num) total key len = 44.443828 GB avg = 0.026 KB total ukey len = 32.886586 GB avg = 0.014 KB total ukey num = 2.325636789 Billion total entry num = 1.705138811 Billion write speed all = 54.268951889 MB/sec (with seq num) write speed all = 53.051537909 MB/sec (without seq num) 2024/11/20-21:10:27.384575 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71387: 924155 keys, 38729906 bytes, temperature: kWarm 2024/11/20-21:10:27.384650 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108227384618, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71387, "file_size": 38729906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1140423497, "table_properties": {"data_size": 32619040, "index_size": 1816384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876340, "raw_average_key_size": 28, "raw_value_size": 108341426, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924155, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108223, "oldest_key_time": 0, "file_creation_time": 1732108223, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71387, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:27.420077 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071388.sst) /dev/shm/ToplingTemp/Topling-993471-hsJ0gN 2024/11/20-21:10:28.541834 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071388.sst): first pass time = 1.12's, 119.653'MB/sec 2024/11/20-21:10:29.876675 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071388.sst): old prealloc_size = 147639500, real_size = 36508640 2024/11/20-21:10:29.877332 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071388.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926797 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.82 avg-zval = 37.18 2024/11/20-21:10:29.877720 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071388.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:30.637802 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071388.sst): second pass time = 1.33's, 81.158'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 50.758'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2890.646'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.366'MB/sec write SST data time = 0.13's, 303.596'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33625.039'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.158'MB/sec zip pipeline throughput = 81.158'MB/sec entries = 926797 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.82 avg-zval = 37.18 usrkeys = 926797 avg-key = 20.00 avg-zkey = 1.97 avg-val = 119.33 avg-zval = 37.18 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 331941 bytes = 1120320 } seq expand size = 2323587 multi value expand size = 0 cnt WriteAppend = 59 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.1747 value = 3.1423 dict = 1.00 all = 3.4743 } Zip/UnZip{ index = 0.0983 value = 0.3182 dict = 1.00 all = 0.2878 } ---------------------------- total value len = 563.749782 GB avg = 0.330 KB (by entry num) total key len = 44.469779 GB avg = 0.026 KB total ukey len = 32.905122 GB avg = 0.014 KB total ukey num = 2.326563586 Billion total entry num = 1.706065608 Billion write speed all = 54.265155659 MB/sec (with seq num) write speed all = 53.047438614 MB/sec (without seq num) 2024/11/20-21:10:30.686506 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71388: 926797 keys, 38631997 bytes, temperature: kWarm 2024/11/20-21:10:30.686568 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108230686535, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71388, "file_size": 38631997, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 978567417, "table_properties": {"data_size": 34454992, "index_size": 1821760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950316, "raw_average_key_size": 28, "raw_value_size": 108267450, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926797, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108227, "oldest_key_time": 0, "file_creation_time": 1732108227, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71388, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:30.728543 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071389.sst) /dev/shm/ToplingTemp/Topling-993471-hvpM6M 2024/11/20-21:10:31.632282 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071389.sst): first pass time = 0.90's, 148.528'MB/sec 2024/11/20-21:10:32.988111 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071389.sst): old prealloc_size = 147639500, real_size = 36420736 2024/11/20-21:10:32.988670 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071389.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.83 avg-zval = 37.09 2024/11/20-21:10:32.988980 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071389.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:33.906727 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071389.sst): second pass time = 1.36's, 79.902'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.593'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3369.337'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.357'MB/sec write SST data time = 0.27's, 142.126'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18153.047'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.902'MB/sec zip pipeline throughput = 79.902'MB/sec entries = 926697 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.83 avg-zval = 37.09 usrkeys = 926697 avg-key = 20.00 avg-zkey = 1.97 avg-val = 118.59 avg-zval = 37.09 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 231949 bytes = 782848 } seq expand size = 1623643 multi value expand size = 0 cnt WriteAppend = 35 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.1776 value = 3.1503 dict = 1.00 all = 3.5130 } Zip/UnZip{ index = 0.0983 value = 0.3174 dict = 1.00 all = 0.2847 } ---------------------------- total value len = 563.859676 GB avg = 0.330 KB (by entry num) total key len = 44.495726 GB avg = 0.026 KB total ukey len = 32.923656 GB avg = 0.014 KB total ukey num = 2.327490283 Billion total entry num = 1.706992305 Billion write speed all = 54.261450029 MB/sec (with seq num) write speed all = 53.043426784 MB/sec (without seq num) 2024/11/20-21:10:33.956762 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71389: 926697 keys, 38206529 bytes, temperature: kWarm 2024/11/20-21:10:33.956862 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108233956811, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71389, "file_size": 38206529, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 963091028, "table_properties": {"data_size": 34367792, "index_size": 1821056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947516, "raw_average_key_size": 28, "raw_value_size": 108270322, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108230, "oldest_key_time": 0, "file_creation_time": 1732108230, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71389, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:34.022996 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071390.sst) /dev/shm/ToplingTemp/Topling-993471-uTu2QO 2024/11/20-21:10:34.937070 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071390.sst): first pass time = 0.91's, 146.851'MB/sec 2024/11/20-21:10:36.201068 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071390.sst): old prealloc_size = 147639500, real_size = 38720496 2024/11/20-21:10:36.201740 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071390.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926131 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0367 GB avg-val = 116.92 avg-zval = 39.66 2024/11/20-21:10:36.202300 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071390.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:37.226037 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071390.sst): second pass time = 1.26's, 85.730'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 40.576'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2753.595'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.27's, 154.932'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32521.564'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.730'MB/sec zip pipeline throughput = 85.730'MB/sec entries = 926131 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.92 avg-zval = 39.66 usrkeys = 926131 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.95 avg-zval = 39.66 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 532896 bytes = 1865152 } seq expand size = 3730272 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0367 GB dict = 1.08 MB all = 0.0416 GB } UnZip/Zip{ index = 10.5139 value = 2.9484 dict = 1.00 all = 3.2272 } Zip/UnZip{ index = 0.0951 value = 0.3392 dict = 1.00 all = 0.3099 } ---------------------------- total value len = 563.971693 GB avg = 0.330 KB (by entry num) total key len = 44.521658 GB avg = 0.026 KB total ukey len = 32.942179 GB avg = 0.014 KB total ukey num = 2.328416414 Billion total entry num = 1.707918436 Billion write speed all = 54.257690516 MB/sec (with seq num) write speed all = 53.039367129 MB/sec (without seq num) 2024/11/20-21:10:37.280266 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71390: 926131 keys, 41589025 bytes, temperature: kWarm 2024/11/20-21:10:37.280358 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108237280311, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71390, "file_size": 41589025, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1140423711, "table_properties": {"data_size": 36727072, "index_size": 1761728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931668, "raw_average_key_size": 28, "raw_value_size": 108286166, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926131, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108234, "oldest_key_time": 0, "file_creation_time": 1732108234, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71390, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:37.328026 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071391.sst) /dev/shm/ToplingTemp/Topling-993471-ALTS1M 2024/11/20-21:10:38.163084 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071391.sst): first pass time = 0.83's, 160.743'MB/sec 2024/11/20-21:10:39.393809 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071391.sst): old prealloc_size = 147639500, real_size = 36216864 2024/11/20-21:10:39.394462 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071391.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922131 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0342 GB avg-val = 117.55 avg-zval = 37.12 2024/11/20-21:10:39.395239 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071391.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:40.283564 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071391.sst): second pass time = 1.23's, 88.137'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 45.213'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2843.914'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.20's, 201.393'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 34687.816'MB/sec zip my value time = 1.23's, unzip length = 0.108'GB zip my value throughput = 88.137'MB/sec zip pipeline throughput = 88.137'MB/sec entries = 922131 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.55 avg-zval = 37.12 usrkeys = 922131 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.07 avg-zval = 37.12 TagRS{ kinds = RS_Key0_TagN bytes = 144200 } TagArray{ size = 859089 bytes = 3006832 } seq expand size = 6013623 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.5132 value = 3.1666 dict = 1.00 all = 3.3365 } Zip/UnZip{ index = 0.0951 value = 0.3158 dict = 1.00 all = 0.2997 } ---------------------------- total value len = 564.086104 GB avg = 0.330 KB (by entry num) total key len = 44.547477 GB avg = 0.026 KB total ukey len = 32.960621 GB avg = 0.014 KB total ukey num = 2.329338545 Billion total entry num = 1.708840567 Billion write speed all = 54.255402852 MB/sec (with seq num) write speed all = 53.036753916 MB/sec (without seq num) 2024/11/20-21:10:40.336546 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71391: 922131 keys, 40226762 bytes, temperature: kWarm 2024/11/20-21:10:40.336620 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108240336582, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71391, "file_size": 40226762, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1140423574, "table_properties": {"data_size": 34231888, "index_size": 1754240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25819668, "raw_average_key_size": 28, "raw_value_size": 108398158, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922131, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108237, "oldest_key_time": 0, "file_creation_time": 1732108237, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71391, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:40.371629 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071392.sst) /dev/shm/ToplingTemp/Topling-993471-tOSa0M 2024/11/20-21:10:41.197188 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071392.sst): first pass time = 0.83's, 162.585'MB/sec 2024/11/20-21:10:42.329955 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071392.sst): old prealloc_size = 147639500, real_size = 36397856 2024/11/20-21:10:42.330509 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071392.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923088 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0344 GB avg-val = 117.40 avg-zval = 37.28 2024/11/20-21:10:42.331297 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071392.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:43.236680 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071392.sst): second pass time = 1.13's, 95.738'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 44.649'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3306.058'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.19's, 213.910'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 19391.594'MB/sec zip my value time = 1.13's, unzip length = 0.108'GB zip my value throughput = 95.738'MB/sec zip pipeline throughput = 95.738'MB/sec entries = 923088 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.40 avg-zval = 37.28 usrkeys = 923088 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 37.28 TagRS{ kinds = RS_Key0_TagN bytes = 144320 } TagArray{ size = 913624 bytes = 3083488 } seq expand size = 6395368 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.5110 value = 3.1494 dict = 1.00 all = 3.3154 } Zip/UnZip{ index = 0.0951 value = 0.3175 dict = 1.00 all = 0.3016 } ---------------------------- total value len = 564.200871 GB avg = 0.330 KB (by entry num) total key len = 44.573324 GB avg = 0.026 KB total ukey len = 32.979083 GB avg = 0.014 KB total ukey num = 2.330261633 Billion total entry num = 1.709763655 Billion write speed all = 54.253655153 MB/sec (with seq num) write speed all = 53.034668823 MB/sec (without seq num) 2024/11/20-21:10:43.286350 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71392: 923088 keys, 40483566 bytes, temperature: kWarm 2024/11/20-21:10:43.286417 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108243286385, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71392, "file_size": 40483566, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1140425140, "table_properties": {"data_size": 34410512, "index_size": 1756416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25846464, "raw_average_key_size": 28, "raw_value_size": 108371304, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923088, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108240, "oldest_key_time": 0, "file_creation_time": 1732108240, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71392, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:43.323539 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071393.sst) /dev/shm/ToplingTemp/Topling-993471-0iZKrQ 2024/11/20-21:10:44.162010 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071393.sst): first pass time = 0.84's, 160.080'MB/sec 2024/11/20-21:10:45.336289 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071393.sst): old prealloc_size = 147639500, real_size = 36537536 2024/11/20-21:10:45.336818 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071393.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923222 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1084 GB zip-val = 0.0346 GB avg-val = 117.38 avg-zval = 37.44 2024/11/20-21:10:45.337553 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071393.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:46.218316 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071393.sst): second pass time = 1.17's, 92.349'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 45.935'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3447.560'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 323.519'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 20343.913'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.349'MB/sec zip pipeline throughput = 92.349'MB/sec entries = 923222 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.38 avg-zval = 37.44 usrkeys = 923222 avg-key = 20.00 avg-zkey = 1.89 avg-val = 124.38 avg-zval = 37.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923222 bytes = 3115888 } seq expand size = 6462554 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0346 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.5812 value = 3.1355 dict = 1.00 all = 3.3128 } Zip/UnZip{ index = 0.0945 value = 0.3189 dict = 1.00 all = 0.3019 } ---------------------------- total value len = 564.315701 GB avg = 0.330 KB (by entry num) total key len = 44.599174 GB avg = 0.026 KB total ukey len = 32.997547 GB avg = 0.014 KB total ukey num = 2.331184855 Billion total entry num = 1.710686877 Billion write speed all = 54.251776655 MB/sec (with seq num) write speed all = 53.032456107 MB/sec (without seq num) 2024/11/20-21:10:46.267367 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71393: 923222 keys, 40514786 bytes, temperature: kWarm 2024/11/20-21:10:46.267440 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108246267406, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71393, "file_size": 40514786, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1063676629, "largest_seqno": 1140424893, "table_properties": {"data_size": 34561520, "index_size": 1745024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25850216, "raw_average_key_size": 28, "raw_value_size": 108367580, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923222, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108243, "oldest_key_time": 0, "file_creation_time": 1732108243, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71393, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:46.304731 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071394.sst) /dev/shm/ToplingTemp/Topling-993471-opbj6N 2024/11/20-21:10:47.125263 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071394.sst): first pass time = 0.82's, 163.581'MB/sec 2024/11/20-21:10:48.287428 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071394.sst): old prealloc_size = 147639500, real_size = 36029664 2024/11/20-21:10:48.287954 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071394.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924315 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 117.21 avg-zval = 36.83 2024/11/20-21:10:48.288725 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071394.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:49.179112 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071394.sst): second pass time = 1.16's, 93.285'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 44.867'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3530.274'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.302'MB/sec write SST data time = 0.13's, 317.296'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33845.066'MB/sec zip my value time = 1.16's, unzip length = 0.108'GB zip my value throughput = 93.285'MB/sec zip pipeline throughput = 93.285'MB/sec entries = 924315 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.21 avg-zval = 36.83 usrkeys = 924315 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.21 avg-zval = 36.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924315 bytes = 3119584 } seq expand size = 6470205 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.5151 value = 3.1826 dict = 1.00 all = 3.3549 } Zip/UnZip{ index = 0.0951 value = 0.3142 dict = 1.00 all = 0.2981 } ---------------------------- total value len = 564.430508 GB avg = 0.330 KB (by entry num) total key len = 44.625055 GB avg = 0.026 KB total ukey len = 33.016034 GB avg = 0.014 KB total ukey num = 2.332109170 Billion total entry num = 1.711611192 Billion write speed all = 54.250000563 MB/sec (with seq num) write speed all = 53.030342933 MB/sec (without seq num) 2024/11/20-21:10:49.227252 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71394: 924315 keys, 40006180 bytes, temperature: kWarm 2024/11/20-21:10:49.227322 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108249227288, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71394, "file_size": 40006180, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1063678867, "largest_seqno": 1140425282, "table_properties": {"data_size": 34040336, "index_size": 1758080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25880820, "raw_average_key_size": 28, "raw_value_size": 108336926, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924315, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108246, "oldest_key_time": 0, "file_creation_time": 1732108246, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71394, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:49.260875 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071395.sst) /dev/shm/ToplingTemp/Topling-993471-jsGSRN 2024/11/20-21:10:50.086950 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071395.sst): first pass time = 0.83's, 162.487'MB/sec 2024/11/20-21:10:51.253757 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071395.sst): old prealloc_size = 147639500, real_size = 36088128 2024/11/20-21:10:51.254308 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071395.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923424 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0341 GB avg-val = 117.35 avg-zval = 36.93 2024/11/20-21:10:51.255113 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071395.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:52.082231 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071395.sst): second pass time = 1.17's, 92.936'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.661'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3340.108'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 312.664'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32932.299'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.936'MB/sec zip pipeline throughput = 92.936'MB/sec entries = 923424 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.35 avg-zval = 36.93 usrkeys = 923424 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.32 avg-zval = 36.93 TagRS{ kinds = RS_Key0_TagN bytes = 144400 } TagArray{ size = 919748 bytes = 3219136 } seq expand size = 6438236 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0403 GB } UnZip/Zip{ index = 10.5103 value = 3.1778 dict = 1.00 all = 3.3294 } Zip/UnZip{ index = 0.0951 value = 0.3147 dict = 1.00 all = 0.3004 } ---------------------------- total value len = 564.545308 GB avg = 0.330 KB (by entry num) total key len = 44.650911 GB avg = 0.026 KB total ukey len = 33.034502 GB avg = 0.014 KB total ukey num = 2.333032594 Billion total entry num = 1.712534616 Billion write speed all = 54.248501059 MB/sec (with seq num) write speed all = 53.028500895 MB/sec (without seq num) 2024/11/20-21:10:52.135374 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71395: 923424 keys, 40313294 bytes, temperature: kWarm 2024/11/20-21:10:52.135445 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108252135413, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71395, "file_size": 40313294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1140423771, "table_properties": {"data_size": 34099888, "index_size": 1757184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25855872, "raw_average_key_size": 28, "raw_value_size": 108361900, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108249, "oldest_key_time": 0, "file_creation_time": 1732108249, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71395, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:52.168921 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071396.sst) /dev/shm/ToplingTemp/Topling-993471-40jgPN 2024/11/20-21:10:52.971975 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071396.sst): first pass time = 0.80's, 167.144'MB/sec 2024/11/20-21:10:54.121256 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071396.sst): old prealloc_size = 147639500, real_size = 35261376 2024/11/20-21:10:54.121829 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071396.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921482 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.65 avg-zval = 36.09 2024/11/20-21:10:54.122570 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071396.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:54.965957 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071396.sst): second pass time = 1.15's, 94.400'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.477'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.343'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.322'MB/sec write SST data time = 0.13's, 303.359'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35589.833'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 94.400'MB/sec zip pipeline throughput = 94.400'MB/sec entries = 921482 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.65 avg-zval = 36.09 usrkeys = 921482 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.15 avg-zval = 36.09 TagRS{ kinds = RS_Key0_TagN bytes = 144080 } TagArray{ size = 855387 bytes = 2993872 } seq expand size = 5987709 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.3778 value = 3.2602 dict = 1.00 all = 3.4188 } Zip/UnZip{ index = 0.0964 value = 0.3067 dict = 1.00 all = 0.2925 } ---------------------------- total value len = 564.659712 GB avg = 0.330 KB (by entry num) total key len = 44.676712 GB avg = 0.026 KB total ukey len = 33.052932 GB avg = 0.014 KB total ukey num = 2.333954076 Billion total entry num = 1.713456098 Billion write speed all = 54.247055958 MB/sec (with seq num) write speed all = 53.026712712 MB/sec (without seq num) 2024/11/20-21:10:55.013843 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71396: 921482 keys, 39258482 bytes, temperature: kWarm 2024/11/20-21:10:55.013914 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108255013882, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71396, "file_size": 39258482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1140424145, "table_properties": {"data_size": 33254960, "index_size": 1775872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25801496, "raw_average_key_size": 28, "raw_value_size": 108416236, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921482, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108252, "oldest_key_time": 0, "file_creation_time": 1732108252, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71396, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:55.058265 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071397.sst) /dev/shm/ToplingTemp/Topling-993471-KU95RP 2024/11/20-21:10:56.063690 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071397.sst): first pass time = 1.01's, 133.504'MB/sec 2024/11/20-21:10:57.363004 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071397.sst): old prealloc_size = 147639500, real_size = 35550320 2024/11/20-21:10:57.363598 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071397.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927359 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0336 GB avg-val = 116.73 avg-zval = 36.18 2024/11/20-21:10:57.364378 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071397.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:10:58.262581 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071397.sst): second pass time = 1.30's, 83.372'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 43.948'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3159.069'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.13's, 306.947'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31436.712'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.372'MB/sec zip pipeline throughput = 83.372'MB/sec entries = 927359 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 36.18 usrkeys = 927359 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 36.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927359 bytes = 3129856 } seq expand size = 6491513 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5148 value = 3.2262 dict = 1.00 all = 3.3950 } Zip/UnZip{ index = 0.0951 value = 0.3100 dict = 1.00 all = 0.2946 } ---------------------------- total value len = 564.774456 GB avg = 0.329 KB (by entry num) total key len = 44.702678 GB avg = 0.026 KB total ukey len = 33.071479 GB avg = 0.014 KB total ukey num = 2.334881435 Billion total entry num = 1.714383457 Billion write speed all = 54.243663014 MB/sec (with seq num) write speed all = 53.023017535 MB/sec (without seq num) 2024/11/20-21:10:58.315207 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71397: 927359 keys, 39534358 bytes, temperature: kWarm 2024/11/20-21:10:58.315278 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108258315246, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71397, "file_size": 39534358, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1053131093, "largest_seqno": 1140424987, "table_properties": {"data_size": 33554400, "index_size": 1763904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966052, "raw_average_key_size": 28, "raw_value_size": 108251754, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927359, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108255, "oldest_key_time": 0, "file_creation_time": 1732108255, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71397, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:10:58.350597 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071398.sst) /dev/shm/ToplingTemp/Topling-993471-rEzVHP 2024/11/20-21:10:59.155441 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071398.sst): first pass time = 0.80's, 166.774'MB/sec 2024/11/20-21:11:00.352143 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071398.sst): old prealloc_size = 147639500, real_size = 36609760 2024/11/20-21:11:00.352725 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071398.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929656 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0346 GB avg-val = 116.37 avg-zval = 37.22 2024/11/20-21:11:00.353324 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071398.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:11:01.127023 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071398.sst): second pass time = 1.20's, 90.473'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 51.069'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3286.997'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.361'MB/sec write SST data time = 0.13's, 302.323'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 20344.885'MB/sec zip my value time = 1.20's, unzip length = 0.108'GB zip my value throughput = 90.473'MB/sec zip pipeline throughput = 90.473'MB/sec entries = 929656 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.37 avg-zval = 37.22 usrkeys = 929656 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.75 avg-zval = 37.22 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 581056 bytes = 1961072 } seq expand size = 4067392 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4465 value = 3.1270 dict = 1.00 all = 3.3917 } Zip/UnZip{ index = 0.0957 value = 0.3198 dict = 1.00 all = 0.2948 } ---------------------------- total value len = 564.886710 GB avg = 0.329 KB (by entry num) total key len = 44.728709 GB avg = 0.026 KB total ukey len = 33.090072 GB avg = 0.014 KB total ukey num = 2.335811091 Billion total entry num = 1.715313113 Billion write speed all = 54.242142139 MB/sec (with seq num) write speed all = 53.021146019 MB/sec (without seq num) 2024/11/20-21:11:01.176236 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71398: 929656 keys, 39572508 bytes, temperature: kWarm 2024/11/20-21:11:01.176306 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108261176272, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71398, "file_size": 39572508, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1140422706, "table_properties": {"data_size": 34597328, "index_size": 1779840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26030368, "raw_average_key_size": 28, "raw_value_size": 108187444, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929656, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108258, "oldest_key_time": 0, "file_creation_time": 1732108258, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71398, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:11:01.212568 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071399.sst) /dev/shm/ToplingTemp/Topling-993471-Pl9LbP 2024/11/20-21:11:02.041186 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071399.sst): first pass time = 0.83's, 161.991'MB/sec 2024/11/20-21:11:03.205423 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071399.sst): old prealloc_size = 147639500, real_size = 36647440 2024/11/20-21:11:03.206013 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071399.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923917 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0347 GB avg-val = 117.27 avg-zval = 37.53 2024/11/20-21:11:03.206798 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071399.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:11:04.016140 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071399.sst): second pass time = 1.16's, 93.122'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 49.816'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3084.348'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.341'MB/sec write SST data time = 0.13's, 304.223'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35305.497'MB/sec zip my value time = 1.16's, unzip length = 0.108'GB zip my value throughput = 93.122'MB/sec zip pipeline throughput = 93.122'MB/sec entries = 923917 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.27 avg-zval = 37.53 usrkeys = 923917 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.41 avg-zval = 37.53 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 809728 bytes = 2732848 } seq expand size = 5668096 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0347 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.5822 value = 3.1251 dict = 1.00 all = 3.3234 } Zip/UnZip{ index = 0.0945 value = 0.3200 dict = 1.00 all = 0.3009 } ---------------------------- total value len = 565.000727 GB avg = 0.329 KB (by entry num) total key len = 44.754578 GB avg = 0.026 KB total ukey len = 33.108550 GB avg = 0.014 KB total ukey num = 2.336735008 Billion total entry num = 1.716237030 Billion write speed all = 54.240645392 MB/sec (with seq num) write speed all = 53.019305574 MB/sec (without seq num) 2024/11/20-21:11:04.078368 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71399: 923917 keys, 40385602 bytes, temperature: kWarm 2024/11/20-21:11:04.078428 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108264078397, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71399, "file_size": 40385602, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1140424648, "table_properties": {"data_size": 34670080, "index_size": 1746176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869676, "raw_average_key_size": 28, "raw_value_size": 108348170, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923917, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108261, "oldest_key_time": 0, "file_creation_time": 1732108261, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71399, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:11:04.115650 140008198919744 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071400.sst) /dev/shm/ToplingTemp/Topling-993471-P9PqaO 2024/11/20-21:11:04.279399 140008198919744 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071400.sst): first pass time = 0.16's, 245.383'MB/sec 2024/11/20-21:11:04.627267 140008198919744 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071400.sst): old prealloc_size = 147639500, real_size = 11529312 2024/11/20-21:11:04.627475 140008198919744 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071400.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 276241 prefix = 4 raw-key = 0.0044 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0324 GB zip-val = 0.0109 GB avg-val = 117.41 avg-zval = 39.58 2024/11/20-21:11:04.627761 140008198919744 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071400.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0007 GB, waited 0.000 sec, Key+Value = 0.040 GB 2024/11/20-21:11:04.874176 140008198919744 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00/071400.sst): second pass time = 0.35's, 93.425'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 51.407'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2844.783'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.336'MB/sec write SST data time = 0.04's, 308.665'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.331'MB, throughput = 10973.615'MB/sec zip my value time = 0.35's, unzip length = 0.032'GB zip my value throughput = 93.425'MB/sec zip pipeline throughput = 93.425'MB/sec entries = 276241 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.41 avg-zval = 39.58 usrkeys = 276241 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.41 avg-zval = 39.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 276241 bytes = 897792 } seq expand size = 1933687 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0055 GB value = 0.0324 GB dict = 0.33 MB all = 0.0402 GB } __ZipSize{ index = 0.0005 GB value = 0.0109 GB dict = 0.33 MB all = 0.0127 GB } UnZip/Zip{ index = 10.4917 value = 2.9665 dict = 1.00 all = 3.1649 } Zip/UnZip{ index = 0.0953 value = 0.3371 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 565.035095 GB avg = 0.329 KB (by entry num) total key len = 44.762313 GB avg = 0.026 KB total ukey len = 33.114075 GB avg = 0.014 KB total ukey num = 2.337011249 Billion total entry num = 1.716513271 Billion write speed all = 54.240250684 MB/sec (with seq num) write speed all = 53.018807511 MB/sec (without seq num) 2024/11/20-21:11:04.895077 140008198919744 [compaction_job.cc:1948] [default] [JOB 13258] Generated table #71400: 276241 keys, 12692071 bytes, temperature: kWarm 2024/11/20-21:11:04.895136 140008198919744 EVENT_LOG_v1 {"time_micros": 1732108264895107, "cf_name": "default", "job": 13258, "event": "table_file_creation", "file_number": 71400, "file_size": 12692071, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1080218124, "largest_seqno": 1140424665, "table_properties": {"data_size": 10933456, "index_size": 526592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7734748, "raw_average_key_size": 28, "raw_value_size": 32434522, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 276241, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108264, "oldest_key_time": 0, "file_creation_time": 1732108264, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 71400, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:11:04.908966 140008198919744 [compaction_job.cc:703] job-13258: subcompact[0], size: 0.688600 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13258/att-00 [071383,071384,071385,071386,071387,071388,071389,071390,071391,071392,071393,071394,071395,071396,071397,071398,071399,071400] 2024/11/20-21:11:05.067112 140008198919744 (Original Log Time 2024/11/20-21:10:07.706757) EVENT_LOG_v1 {"time_micros": 1732108207706700, "job": 13258, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [71375], "files_L3": [71326, 71327, 71328, 71329, 71330, 71331, 71332, 71333, 71334, 71335, 71336, 71337, 71338, 71271, 71272, 71273, 71274], "score": 0.908323, "input_data_size": 734371693, "oldest_snapshot_seqno": 1140919760} 2024/11/20-21:11:05.067116 140008198919744 (Original Log Time 2024/11/20-21:11:05.047881) [compaction_job.cc:2022] [default] [JOB 13258] Compacted 1@2 + 17@3 files to L3 => 688600226 bytes 2024/11/20-21:11:05.067117 140008198919744 (Original Log Time 2024/11/20-21:11:05.067021) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 18 0 0 0] max score 0.03, MB/sec: 12.8 rd, 12.0 wr, level 3, files in(1, 17) out(18 +0 blob) MB in(54.3, 646.0 +0.0 blob) out(656.7 +0.0 blob), read-write-amplify(25.0) write-amplify(12.1) OK, records in: 16186391, records dropped: 189718 output_compression: Snappy 2024/11/20-21:11:05.067122 140008198919744 (Original Log Time 2024/11/20-21:11:05.067089) EVENT_LOG_v1 {"time_micros": 1732108265067059, "cf": "default", "job": 13258, "event": "compaction_finished", "compaction_time_micros": 57202215, "compaction_time_cpu_micros": 25728486, "output_level": 3, "num_output_files": 18, "total_output_size": 688600226, "num_input_records": 16186391, "num_output_records": 15996673, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 558431443, "file_range_sync_nanos": 0, "file_fsync_nanos": 16420, "file_prepare_write_nanos": 62228, "lsm_state": [0, 0, 0, 18, 0, 0, 0]} 2024/11/20-21:11:05.200325 140008198919744 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732108207,"executesMs":57462,"compactionJobId":13258,"attempt":0,"compactionInputRawBytes":2348811558,"compactionInputZipBytes":734371693,"compactionOutputRawBytes":0,"compactionOutputZipBytes":688600226,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-e5f7872ae5b300c5bd09fb272c59a490-eb59804942d51ff0-00"}}, response = 2024/11/20-21:11:05.200403 140008198919744 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-13258/att-00: olev 3, work 57.463 s, result 8.037 ms, install 19.257 ms, input{raw 2.188 GiB zip 700.351 MiB}