2024/11/20-13:10:52.095524 140257835828800 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00: fork to child time = 0.021924 sec 2024/11/20-13:10:52.096224 140257835828800 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00 2024/11/20-13:10:52.103565 140257835828800 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00 2024/11/20-13:10:52.105753 140257835828800 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-13:10:52.106087 140257835828800 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-13:10:52.157024 140257835828800 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-06744/att-00/MANIFEST-019309 2024/11/20-13:10:52.214215 140257835828800 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-06744/att-00/MANIFEST-019309 succeeded,manifest_file_number is 19309, next_file_number is 29094, last_sequence is 557489231, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 28899 2024/11/20-13:10:52.214233 140257835828800 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-13:10:52.214395 140257835828800 [version_set.cc:5724] Creating manifest 29094 2024/11/20-13:10:52.245814 140257835828800 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-13:10:52.265176 140257835828800 [compaction_job.cc:2437] [default] [JOB 6744] Compacting 1@2 + 22@3 files to L3, score 0.79, subcompactions 1 : 1 2024/11/20-13:10:52.265206 140257835828800 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 6253 Base level 2, inputs: [29070(12MB)], [29091(40MB) 29092(847KB) 28990(39MB) 29019(39MB) 29016(39MB) 29017(39MB) 29018(39MB) 29020(39MB) 29021(39MB) 29022(39MB) 29023(39MB) 29024(39MB) 29025(39MB) 29029(39MB) 29027(39MB) 29028(39MB) 29026(39MB) 29030(39MB) 29033(39MB) 29032(39MB) 29034(39MB) 29031(14MB)] 2024/11/20-13:10:52.270923 140257835828800 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-13:10:52.270957 140257835828800 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-13:10:52.270984 140257835828800 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-13:10:52.270995 140257835828800 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-13:10:52.271011 140257835828800 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-13:10:52.769330 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029095.sst) /dev/shm/ToplingTemp/Topling-219654-gMEiYp 2024/11/20-13:10:52.937029 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029095.sst): first pass time = 0.17's, 800.561'MB/sec 2024/11/20-13:10:53.457678 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029095.sst): old prealloc_size = 147639500, real_size = 40676656 2024/11/20-13:10:53.457809 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029095.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129424 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.04 avg-zval = 312.32 2024/11/20-13:10:53.457937 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029095.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:10:53.638919 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029095.sst): second pass time = 0.52's, 253.183'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 233.116'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2024.859'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.06's, 686.818'MB/sec dict compress time = 2827527657.54's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46342.066'MB/sec zip my value time = 0.52's, unzip length = 0.132'GB zip my value throughput = 253.183'MB/sec zip pipeline throughput = 253.183'MB/sec entries = 129424 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.04 avg-zval = 312.32 usrkeys = 129424 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1020.64 avg-zval = 312.32 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 66522 bytes = 199584 } seq expand size = 465654 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.9793 value = 3.2564 dict = 1.00 all = 3.1772 } Zip/UnZip{ index = 0.1433 value = 0.3071 dict = 1.00 all = 0.3147 } ---------------------------- total value len = 0.132095 GB avg = 1.021 KB (by entry num) total key len = 0.002588 GB avg = 0.020 KB total ukey len = 0.001553 GB avg = 0.012 KB total ukey num = 0.000129424 Billion total entry num = 0.000129424 Billion write speed all = 0.008528715 MB/sec (with seq num) write speed all = 0.008463150 MB/sec (without seq num) 2024/11/20-13:10:53.694527 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29095: 129424 keys, 42244046 bytes, temperature: kWarm 2024/11/20-13:10:53.694649 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079453694592, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29095, "file_size": 42244046, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557334186, "table_properties": {"data_size": 40421600, "index_size": 222528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588480, "raw_average_key_size": 20, "raw_value_size": 131629633, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079452, "oldest_key_time": 0, "file_creation_time": 1732079452, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29095, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:10:53.765674 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029096.sst) /dev/shm/ToplingTemp/Topling-219654-FwHR5p 2024/11/20-13:10:54.358156 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029096.sst): first pass time = 0.59's, 226.565'MB/sec 2024/11/20-13:10:54.905032 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029096.sst): old prealloc_size = 147639500, real_size = 40354896 2024/11/20-13:10:54.905168 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029096.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.84 2024/11/20-13:10:54.905216 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029096.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:10:55.075121 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029096.sst): second pass time = 0.55's, 241.008'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 245.391'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2313.391'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.283'MB/sec write SST data time = 0.06's, 747.837'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 27048.860'MB/sec zip my value time = 0.55's, unzip length = 0.132'GB zip my value throughput = 241.008'MB/sec zip pipeline throughput = 241.008'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.00 avg-zval = 309.84 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1017.03 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 557 bytes = 1680 } seq expand size = 3899 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0753 value = 3.2823 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1413 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.263729 GB avg = 1.019 KB (by entry num) total key len = 0.005177 GB avg = 0.020 KB total ukey len = 0.003106 GB avg = 0.012 KB total ukey num = 0.000258854 Billion total entry num = 0.000258854 Billion write speed all = 0.017026665 MB/sec (with seq num) write speed all = 0.016895544 MB/sec (without seq num) 2024/11/20-13:10:55.127201 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29096: 129430 keys, 41724135 bytes, temperature: kWarm 2024/11/20-13:10:55.127292 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079455127246, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29096, "file_size": 41724135, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557327061, "table_properties": {"data_size": 40102848, "index_size": 219520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629889, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079453, "oldest_key_time": 0, "file_creation_time": 1732079453, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29096, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:10:55.193747 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029097.sst) /dev/shm/ToplingTemp/Topling-219654-fEIwaq 2024/11/20-13:10:55.860682 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029097.sst): first pass time = 0.67's, 201.266'MB/sec 2024/11/20-13:10:56.360471 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029097.sst): old prealloc_size = 147639500, real_size = 40354816 2024/11/20-13:10:56.360586 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029097.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.86 2024/11/20-13:10:56.360638 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029097.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:10:56.518809 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029097.sst): second pass time = 0.50's, 263.742'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.532'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2365.247'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.279'MB/sec write SST data time = 0.04's, 986.688'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47070.540'MB/sec zip my value time = 0.50's, unzip length = 0.132'GB zip my value throughput = 263.742'MB/sec zip pipeline throughput = 263.742'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.86 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.04 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 67 bytes = 208 } seq expand size = 469 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0957 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.395359 GB avg = 1.018 KB (by entry num) total key len = 0.007766 GB avg = 0.020 KB total ukey len = 0.004659 GB avg = 0.012 KB total ukey num = 0.000388279 Billion total entry num = 0.000388279 Billion write speed all = 0.025522815 MB/sec (with seq num) write speed all = 0.025326152 MB/sec (without seq num) 2024/11/20-13:10:56.612203 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29097: 129425 keys, 41722646 bytes, temperature: kWarm 2024/11/20-13:10:56.612260 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079456612230, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29097, "file_size": 41722646, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557290855, "table_properties": {"data_size": 40103408, "index_size": 218880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629573, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079455, "oldest_key_time": 0, "file_creation_time": 1732079455, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29097, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:10:56.650913 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029098.sst) /dev/shm/ToplingTemp/Topling-219654-IFzZto 2024/11/20-13:10:57.176395 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029098.sst): first pass time = 0.53's, 255.435'MB/sec 2024/11/20-13:10:57.676100 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029098.sst): old prealloc_size = 147639500, real_size = 40354816 2024/11/20-13:10:57.676211 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029098.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.86 2024/11/20-13:10:57.676258 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029098.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:10:57.833065 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029098.sst): second pass time = 0.50's, 263.760'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.850'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2434.725'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.282'MB/sec write SST data time = 0.04's, 994.388'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46751.162'MB/sec zip my value time = 0.50's, unzip length = 0.132'GB zip my value throughput = 263.760'MB/sec zip pipeline throughput = 263.760'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.86 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 11 bytes = 48 } seq expand size = 77 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.526989 GB avg = 1.018 KB (by entry num) total key len = 0.010354 GB avg = 0.020 KB total ukey len = 0.006212 GB avg = 0.012 KB total ukey num = 0.000517705 Billion total entry num = 0.000517705 Billion write speed all = 0.034017675 MB/sec (with seq num) write speed all = 0.033755480 MB/sec (without seq num) 2024/11/20-13:10:57.884108 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29098: 129426 keys, 41722659 bytes, temperature: kWarm 2024/11/20-13:10:57.884164 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079457884135, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29098, "file_size": 41722659, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557310022, "table_properties": {"data_size": 40103472, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629719, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079456, "oldest_key_time": 0, "file_creation_time": 1732079456, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29098, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:10:57.924368 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029099.sst) /dev/shm/ToplingTemp/Topling-219654-wWqqnp 2024/11/20-13:10:58.490177 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029099.sst): first pass time = 0.57's, 237.232'MB/sec 2024/11/20-13:10:59.195247 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029099.sst): old prealloc_size = 147639500, real_size = 40354704 2024/11/20-13:10:59.195365 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029099.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.85 2024/11/20-13:10:59.195413 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029099.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:10:59.350833 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029099.sst): second pass time = 0.70's, 186.850'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.185'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2303.593'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.285'MB/sec write SST data time = 0.04's, 992.682'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 38701.255'MB/sec zip my value time = 0.70's, unzip length = 0.132'GB zip my value throughput = 186.850'MB/sec zip pipeline throughput = 226.343'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.85 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 21 bytes = 64 } seq expand size = 147 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.658619 GB avg = 1.018 KB (by entry num) total key len = 0.012943 GB avg = 0.020 KB total ukey len = 0.007766 GB avg = 0.012 KB total ukey num = 0.000647135 Billion total entry num = 0.000647135 Billion write speed all = 0.042510611 MB/sec (with seq num) write speed all = 0.042182896 MB/sec (without seq num) 2024/11/20-13:10:59.400006 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29099: 129430 keys, 41722560 bytes, temperature: kWarm 2024/11/20-13:10:59.400080 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079459400041, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29099, "file_size": 41722560, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557325266, "table_properties": {"data_size": 40103360, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131630142, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079457, "oldest_key_time": 0, "file_creation_time": 1732079457, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29099, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:10:59.434180 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029100.sst) /dev/shm/ToplingTemp/Topling-219654-0nNq4q 2024/11/20-13:11:00.199655 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029100.sst): first pass time = 0.77's, 175.348'MB/sec 2024/11/20-13:11:00.744523 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029100.sst): old prealloc_size = 147639500, real_size = 40352960 2024/11/20-13:11:00.744639 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029100.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.84 2024/11/20-13:11:00.744692 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029100.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:00.900493 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029100.sst): second pass time = 0.54's, 241.920'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.498'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2468.843'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.282'MB/sec write SST data time = 0.04's, 1012.243'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46978.111'MB/sec zip my value time = 0.54's, unzip length = 0.132'GB zip my value throughput = 241.920'MB/sec zip pipeline throughput = 241.920'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 19 bytes = 64 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2824 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 0.790249 GB avg = 1.018 KB (by entry num) total key len = 0.015531 GB avg = 0.020 KB total ukey len = 0.009319 GB avg = 0.012 KB total ukey num = 0.000776563 Billion total entry num = 0.000776563 Billion write speed all = 0.051001765 MB/sec (with seq num) write speed all = 0.050608545 MB/sec (without seq num) 2024/11/20-13:11:00.951797 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29100: 129428 keys, 41720587 bytes, temperature: kWarm 2024/11/20-13:11:00.951851 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079460951823, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29100, "file_size": 41720587, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557267434, "table_properties": {"data_size": 40101616, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629445, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079459, "oldest_key_time": 0, "file_creation_time": 1732079459, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29100, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:00.986330 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029101.sst) /dev/shm/ToplingTemp/Topling-219654-llkg4m 2024/11/20-13:11:01.666078 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029101.sst): first pass time = 0.68's, 197.462'MB/sec 2024/11/20-13:11:02.591199 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029101.sst): old prealloc_size = 147639500, real_size = 40350928 2024/11/20-13:11:02.591310 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029101.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.97 avg-zval = 309.81 2024/11/20-13:11:02.591360 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029101.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:02.749486 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029101.sst): second pass time = 0.92's, 142.396'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.593'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2388.456'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.281'MB/sec write SST data time = 0.04's, 965.548'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39817.328'MB/sec zip my value time = 0.92's, unzip length = 0.132'GB zip my value throughput = 142.396'MB/sec zip pipeline throughput = 237.649'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.81 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.81 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 14 bytes = 48 } seq expand size = 98 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0982 value = 3.2826 dict = 1.00 all = 3.2172 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 0.921878 GB avg = 1.018 KB (by entry num) total key len = 0.018120 GB avg = 0.020 KB total ukey len = 0.010872 GB avg = 0.012 KB total ukey num = 0.000905996 Billion total entry num = 0.000905996 Billion write speed all = 0.059490132 MB/sec (with seq num) write speed all = 0.059031427 MB/sec (without seq num) 2024/11/20-13:11:02.810332 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29101: 129433 keys, 41718470 bytes, temperature: kWarm 2024/11/20-13:11:02.810395 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079462810365, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29101, "file_size": 41718470, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557329138, "table_properties": {"data_size": 40099584, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629482, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079460, "oldest_key_time": 0, "file_creation_time": 1732079460, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29101, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:02.846350 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029102.sst) /dev/shm/ToplingTemp/Topling-219654-4YQndn 2024/11/20-13:11:03.396146 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029102.sst): first pass time = 0.55's, 244.146'MB/sec 2024/11/20-13:11:03.943531 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029102.sst): old prealloc_size = 147639500, real_size = 40351136 2024/11/20-13:11:03.943650 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029102.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.82 2024/11/20-13:11:03.943702 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029102.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:04.099734 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029102.sst): second pass time = 0.55's, 240.741'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.113'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2178.921'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.285'MB/sec write SST data time = 0.04's, 980.524'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47020.064'MB/sec zip my value time = 0.55's, unzip length = 0.132'GB zip my value throughput = 240.741'MB/sec zip pipeline throughput = 252.745'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.82 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.82 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 28 bytes = 96 } seq expand size = 196 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2825 dict = 1.00 all = 3.2172 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.053508 GB avg = 1.017 KB (by entry num) total key len = 0.020709 GB avg = 0.020 KB total ukey len = 0.012425 GB avg = 0.012 KB total ukey num = 0.001035426 Billion total entry num = 0.001035426 Billion write speed all = 0.067978662 MB/sec (with seq num) write speed all = 0.067454471 MB/sec (without seq num) 2024/11/20-13:11:04.150170 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29102: 129430 keys, 41718897 bytes, temperature: kWarm 2024/11/20-13:11:04.150236 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079464150205, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29102, "file_size": 41718897, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557318437, "table_properties": {"data_size": 40099792, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629499, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079462, "oldest_key_time": 0, "file_creation_time": 1732079462, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29102, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:04.185919 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029103.sst) /dev/shm/ToplingTemp/Topling-219654-XKYhop 2024/11/20-13:11:05.000571 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029103.sst): first pass time = 0.81's, 164.764'MB/sec 2024/11/20-13:11:05.538629 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029103.sst): old prealloc_size = 147639500, real_size = 40354624 2024/11/20-13:11:05.538747 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029103.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.85 2024/11/20-13:11:05.538824 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029103.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:05.701618 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029103.sst): second pass time = 0.54's, 245.015'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 256.004'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2322.271'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.04's, 954.842'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 24985.439'MB/sec zip my value time = 0.54's, unzip length = 0.132'GB zip my value throughput = 245.015'MB/sec zip pipeline throughput = 245.015'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 22 bytes = 80 } seq expand size = 154 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.185138 GB avg = 1.017 KB (by entry num) total key len = 0.023297 GB avg = 0.020 KB total ukey len = 0.013978 GB avg = 0.012 KB total ukey num = 0.001164854 Billion total entry num = 0.001164854 Billion write speed all = 0.076464517 MB/sec (with seq num) write speed all = 0.075874862 MB/sec (without seq num) 2024/11/20-13:11:05.751744 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29103: 129428 keys, 41722255 bytes, temperature: kWarm 2024/11/20-13:11:05.751831 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079465751801, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29103, "file_size": 41722255, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557130467, "table_properties": {"data_size": 40103280, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629470, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079464, "oldest_key_time": 0, "file_creation_time": 1732079464, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29103, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:05.788004 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029104.sst) /dev/shm/ToplingTemp/Topling-219654-ybgY5m 2024/11/20-13:11:06.304643 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029104.sst): first pass time = 0.52's, 259.811'MB/sec 2024/11/20-13:11:06.771144 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029104.sst): old prealloc_size = 147639500, real_size = 40355504 2024/11/20-13:11:06.771263 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029104.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.86 2024/11/20-13:11:06.771319 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029104.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:06.988467 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029104.sst): second pass time = 0.47's, 282.628'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 192.007'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2261.641'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.187'MB/sec write SST data time = 0.04's, 951.329'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 33124.438'MB/sec zip my value time = 0.47's, unzip length = 0.132'GB zip my value throughput = 282.628'MB/sec zip pipeline throughput = 282.628'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.86 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2822 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.316767 GB avg = 1.017 KB (by entry num) total key len = 0.025886 GB avg = 0.020 KB total ukey len = 0.015531 GB avg = 0.012 KB total ukey num = 0.001294280 Billion total entry num = 0.001294280 Billion write speed all = 0.084950344 MB/sec (with seq num) write speed all = 0.084295226 MB/sec (without seq num) 2024/11/20-13:11:07.037230 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29104: 129426 keys, 41723885 bytes, temperature: kWarm 2024/11/20-13:11:07.037295 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079467037266, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29104, "file_size": 41723885, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557211533, "table_properties": {"data_size": 40104160, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629520, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079465, "oldest_key_time": 0, "file_creation_time": 1732079465, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29104, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:07.073116 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029105.sst) /dev/shm/ToplingTemp/Topling-219654-c6Tbyq 2024/11/20-13:11:07.816409 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029105.sst): first pass time = 0.74's, 180.584'MB/sec 2024/11/20-13:11:08.371779 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029105.sst): old prealloc_size = 147639500, real_size = 40353600 2024/11/20-13:11:08.371905 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029105.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.84 2024/11/20-13:11:08.371964 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029105.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:08.527178 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029105.sst): second pass time = 0.55's, 237.347'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.483'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2155.547'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.279'MB/sec write SST data time = 0.04's, 1058.101'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 26934.540'MB/sec zip my value time = 0.55's, unzip length = 0.132'GB zip my value throughput = 237.347'MB/sec zip pipeline throughput = 237.347'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.84 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2824 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.448397 GB avg = 1.017 KB (by entry num) total key len = 0.028474 GB avg = 0.020 KB total ukey len = 0.017085 GB avg = 0.012 KB total ukey num = 0.001423709 Billion total entry num = 0.001423709 Billion write speed all = 0.093433321 MB/sec (with seq num) write speed all = 0.092712761 MB/sec (without seq num) 2024/11/20-13:11:08.580831 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29105: 129429 keys, 41721761 bytes, temperature: kWarm 2024/11/20-13:11:08.580897 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079468580866, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29105, "file_size": 41721761, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557120169, "table_properties": {"data_size": 40102256, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629787, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079467, "oldest_key_time": 0, "file_creation_time": 1732079467, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29105, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:08.616357 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029106.sst) /dev/shm/ToplingTemp/Topling-219654-KMBNLn 2024/11/20-13:11:09.187507 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029106.sst): first pass time = 0.57's, 235.011'MB/sec 2024/11/20-13:11:09.686635 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029106.sst): old prealloc_size = 147639500, real_size = 40353504 2024/11/20-13:11:09.686760 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029106.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.84 2024/11/20-13:11:09.686824 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029106.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:09.843136 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029106.sst): second pass time = 0.50's, 264.140'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.615'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1967.628'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.280'MB/sec write SST data time = 0.04's, 1017.609'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 26571.555'MB/sec zip my value time = 0.50's, unzip length = 0.132'GB zip my value throughput = 264.140'MB/sec zip pipeline throughput = 264.140'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.580027 GB avg = 1.017 KB (by entry num) total key len = 0.031063 GB avg = 0.020 KB total ukey len = 0.018638 GB avg = 0.012 KB total ukey num = 0.001553137 Billion total entry num = 0.001553137 Billion write speed all = 0.101916041 MB/sec (with seq num) write speed all = 0.101130041 MB/sec (without seq num) 2024/11/20-13:11:09.892653 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29106: 129428 keys, 41721285 bytes, temperature: kWarm 2024/11/20-13:11:09.892719 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079469892688, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29106, "file_size": 41721285, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557190405, "table_properties": {"data_size": 40102160, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629182, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079468, "oldest_key_time": 0, "file_creation_time": 1732079468, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29106, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:09.927437 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029107.sst) /dev/shm/ToplingTemp/Topling-219654-ZRLkFq 2024/11/20-13:11:10.543694 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029107.sst): first pass time = 0.62's, 217.812'MB/sec 2024/11/20-13:11:11.429165 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029107.sst): old prealloc_size = 147639500, real_size = 40353488 2024/11/20-13:11:11.429272 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029107.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.84 2024/11/20-13:11:11.429319 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029107.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:11.582602 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029107.sst): second pass time = 0.88's, 148.776'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 271.938'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2456.095'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.285'MB/sec write SST data time = 0.04's, 1043.226'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 19491.468'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 148.776'MB/sec zip pipeline throughput = 268.697'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 22 bytes = 80 } seq expand size = 154 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2824 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.711656 GB avg = 1.017 KB (by entry num) total key len = 0.033651 GB avg = 0.020 KB total ukey len = 0.020191 GB avg = 0.012 KB total ukey num = 0.001682565 Billion total entry num = 0.001682565 Billion write speed all = 0.110394437 MB/sec (with seq num) write speed all = 0.109543030 MB/sec (without seq num) 2024/11/20-13:11:11.631563 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29107: 129428 keys, 41720849 bytes, temperature: kWarm 2024/11/20-13:11:11.631627 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079471631596, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29107, "file_size": 41720849, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557280235, "table_properties": {"data_size": 40102144, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629835, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079469, "oldest_key_time": 0, "file_creation_time": 1732079469, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29107, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:11.666100 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029108.sst) /dev/shm/ToplingTemp/Topling-219654-fXy9Gn 2024/11/20-13:11:12.224069 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029108.sst): first pass time = 0.56's, 240.565'MB/sec 2024/11/20-13:11:12.719366 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029108.sst): old prealloc_size = 147639500, real_size = 40354432 2024/11/20-13:11:12.719478 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029108.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.85 2024/11/20-13:11:12.719525 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029108.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:12.873100 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029108.sst): second pass time = 0.49's, 266.136'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 271.438'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2405.840'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.285'MB/sec write SST data time = 0.04's, 1043.464'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47783.968'MB/sec zip my value time = 0.49's, unzip length = 0.132'GB zip my value throughput = 266.136'MB/sec zip pipeline throughput = 266.136'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.85 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 16 bytes = 64 } seq expand size = 112 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.843287 GB avg = 1.017 KB (by entry num) total key len = 0.036240 GB avg = 0.020 KB total ukey len = 0.021744 GB avg = 0.012 KB total ukey num = 0.001811991 Billion total entry num = 0.001811991 Billion write speed all = 0.118874360 MB/sec (with seq num) write speed all = 0.117957536 MB/sec (without seq num) 2024/11/20-13:11:12.922116 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29108: 129426 keys, 41722424 bytes, temperature: kWarm 2024/11/20-13:11:12.922180 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079472922151, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29108, "file_size": 41722424, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557136087, "table_properties": {"data_size": 40103088, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131630180, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079471, "oldest_key_time": 0, "file_creation_time": 1732079471, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29108, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:12.958001 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029109.sst) /dev/shm/ToplingTemp/Topling-219654-hTf81m 2024/11/20-13:11:13.776588 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029109.sst): first pass time = 0.82's, 163.972'MB/sec 2024/11/20-13:11:14.387476 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029109.sst): old prealloc_size = 147639500, real_size = 40353312 2024/11/20-13:11:14.387620 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029109.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.84 2024/11/20-13:11:14.387686 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029109.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:14.579210 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029109.sst): second pass time = 0.61's, 215.739'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 217.604'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1944.253'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.217'MB/sec write SST data time = 0.04's, 976.110'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43916.393'MB/sec zip my value time = 0.61's, unzip length = 0.132'GB zip my value throughput = 215.739'MB/sec zip pipeline throughput = 215.739'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.84 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2824 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.974917 GB avg = 1.017 KB (by entry num) total key len = 0.038828 GB avg = 0.020 KB total ukey len = 0.023297 GB avg = 0.012 KB total ukey num = 0.001941420 Billion total entry num = 0.001941420 Billion write speed all = 0.127349527 MB/sec (with seq num) write speed all = 0.126367322 MB/sec (without seq num) 2024/11/20-13:11:14.628451 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29109: 129429 keys, 41721355 bytes, temperature: kWarm 2024/11/20-13:11:14.628506 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079474628478, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29109, "file_size": 41721355, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557309903, "table_properties": {"data_size": 40101968, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629770, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079472, "oldest_key_time": 0, "file_creation_time": 1732079472, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29109, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:14.661576 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029110.sst) /dev/shm/ToplingTemp/Topling-219654-953nYo 2024/11/20-13:11:15.136073 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029110.sst): first pass time = 0.47's, 282.884'MB/sec 2024/11/20-13:11:15.623339 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029110.sst): old prealloc_size = 147639500, real_size = 40353872 2024/11/20-13:11:15.623457 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029110.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.84 2024/11/20-13:11:15.623513 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029110.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:15.791761 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029110.sst): second pass time = 0.49's, 270.574'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 247.788'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2291.411'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.282'MB/sec write SST data time = 0.05's, 779.926'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35699.671'MB/sec zip my value time = 0.49's, unzip length = 0.132'GB zip my value throughput = 270.574'MB/sec zip pipeline throughput = 270.574'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.106546 GB avg = 1.017 KB (by entry num) total key len = 0.041417 GB avg = 0.020 KB total ukey len = 0.024850 GB avg = 0.012 KB total ukey num = 0.002070848 Billion total entry num = 0.002070848 Billion write speed all = 0.135827103 MB/sec (with seq num) write speed all = 0.134779498 MB/sec (without seq num) 2024/11/20-13:11:15.846426 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29110: 129428 keys, 41722015 bytes, temperature: kWarm 2024/11/20-13:11:15.846519 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079475846474, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29110, "file_size": 41722015, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557049296, "table_properties": {"data_size": 40102528, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629722, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079474, "oldest_key_time": 0, "file_creation_time": 1732079474, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29110, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:15.909868 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029111.sst) /dev/shm/ToplingTemp/Topling-219654-BNLD4m 2024/11/20-13:11:16.618870 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029111.sst): first pass time = 0.71's, 189.320'MB/sec 2024/11/20-13:11:17.110543 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029111.sst): old prealloc_size = 147639500, real_size = 40352576 2024/11/20-13:11:17.110671 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029111.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.83 2024/11/20-13:11:17.110731 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029111.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:17.328056 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029111.sst): second pass time = 0.49's, 269.885'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 191.838'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2399.377'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.202'MB/sec write SST data time = 0.06's, 727.637'MB/sec dict compress time = 0.00's, 1378288.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45017.082'MB/sec zip my value time = 0.49's, unzip length = 0.132'GB zip my value throughput = 269.885'MB/sec zip pipeline throughput = 269.885'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.83 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.83 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2824 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 2.238177 GB avg = 1.017 KB (by entry num) total key len = 0.044006 GB avg = 0.020 KB total ukey len = 0.026403 GB avg = 0.012 KB total ukey num = 0.002200277 Billion total entry num = 0.002200277 Billion write speed all = 0.144300439 MB/sec (with seq num) write speed all = 0.143187466 MB/sec (without seq num) 2024/11/20-13:11:17.379083 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29111: 129429 keys, 41720623 bytes, temperature: kWarm 2024/11/20-13:11:17.379148 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079477379118, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29111, "file_size": 41720623, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557321817, "table_properties": {"data_size": 40101232, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629925, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079475, "oldest_key_time": 0, "file_creation_time": 1732079475, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29111, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:17.411791 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029112.sst) /dev/shm/ToplingTemp/Topling-219654-H4GGup 2024/11/20-13:11:18.085317 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029112.sst): first pass time = 0.67's, 199.290'MB/sec 2024/11/20-13:11:18.578425 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029112.sst): old prealloc_size = 147639500, real_size = 40350864 2024/11/20-13:11:18.578571 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029112.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.82 2024/11/20-13:11:18.578628 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029112.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:18.734074 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029112.sst): second pass time = 0.49's, 267.365'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.050'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2045.009'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.283'MB/sec write SST data time = 0.04's, 1016.799'MB/sec dict compress time = 0.00's, 1377966.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44529.520'MB/sec zip my value time = 0.49's, unzip length = 0.132'GB zip my value throughput = 267.365'MB/sec zip pipeline throughput = 267.365'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.82 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.82 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2826 dict = 1.00 all = 3.2172 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 2.369806 GB avg = 1.017 KB (by entry num) total key len = 0.046594 GB avg = 0.020 KB total ukey len = 0.027956 GB avg = 0.012 KB total ukey num = 0.002329707 Billion total entry num = 0.002329707 Billion write speed all = 0.152773351 MB/sec (with seq num) write speed all = 0.151595013 MB/sec (without seq num) 2024/11/20-13:11:18.786063 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29112: 129430 keys, 41718573 bytes, temperature: kWarm 2024/11/20-13:11:18.786126 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079478786096, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29112, "file_size": 41718573, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557322829, "table_properties": {"data_size": 40099520, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629330, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079477, "oldest_key_time": 0, "file_creation_time": 1732079477, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29112, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:18.818725 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029113.sst) /dev/shm/ToplingTemp/Topling-219654-PAAfSo 2024/11/20-13:11:19.373593 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029113.sst): first pass time = 0.55's, 241.916'MB/sec 2024/11/20-13:11:19.902315 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029113.sst): old prealloc_size = 147639500, real_size = 40354416 2024/11/20-13:11:19.902431 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029113.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.85 2024/11/20-13:11:19.902484 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029113.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:20.055308 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029113.sst): second pass time = 0.53's, 249.348'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 272.736'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2345.848'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.287'MB/sec write SST data time = 0.04's, 1043.843'MB/sec dict compress time = 0.00's, 1378119.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44321.059'MB/sec zip my value time = 0.53's, unzip length = 0.132'GB zip my value throughput = 249.348'MB/sec zip pipeline throughput = 249.348'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 19 bytes = 64 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.501436 GB avg = 1.017 KB (by entry num) total key len = 0.049183 GB avg = 0.020 KB total ukey len = 0.029510 GB avg = 0.012 KB total ukey num = 0.002459134 Billion total entry num = 0.002459134 Billion write speed all = 0.161245652 MB/sec (with seq num) write speed all = 0.160001955 MB/sec (without seq num) 2024/11/20-13:11:20.104036 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29113: 129427 keys, 41722310 bytes, temperature: kWarm 2024/11/20-13:11:20.104109 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079480104078, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29113, "file_size": 41722310, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557260489, "table_properties": {"data_size": 40103072, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629864, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079478, "oldest_key_time": 0, "file_creation_time": 1732079478, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29113, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:20.138644 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029114.sst) /dev/shm/ToplingTemp/Topling-219654-Hi6Rdo 2024/11/20-13:11:20.841798 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029114.sst): first pass time = 0.70's, 190.890'MB/sec 2024/11/20-13:11:21.319501 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029114.sst): old prealloc_size = 147639500, real_size = 40353824 2024/11/20-13:11:21.319633 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029114.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.84 2024/11/20-13:11:21.319690 140257835828800 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029114.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:11:21.471904 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029114.sst): second pass time = 0.48's, 276.008'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 273.775'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2145.297'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.286'MB/sec write SST data time = 0.04's, 1065.490'MB/sec dict compress time = 0.00's, 1378279.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 34364.192'MB/sec zip my value time = 0.48's, unzip length = 0.132'GB zip my value throughput = 276.008'MB/sec zip pipeline throughput = 276.008'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 15 bytes = 48 } seq expand size = 105 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 2.633066 GB avg = 1.017 KB (by entry num) total key len = 0.051771 GB avg = 0.020 KB total ukey len = 0.031063 GB avg = 0.012 KB total ukey num = 0.002588562 Billion total entry num = 0.002588562 Billion write speed all = 0.169715521 MB/sec (with seq num) write speed all = 0.168406484 MB/sec (without seq num) 2024/11/20-13:11:21.520728 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29114: 129428 keys, 41721830 bytes, temperature: kWarm 2024/11/20-13:11:21.520795 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079481520760, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29114, "file_size": 41721830, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557250321, "table_properties": {"data_size": 40102480, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629986, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732079480, "oldest_key_time": 0, "file_creation_time": 1732079480, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29114, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:21.553223 140257835828800 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029115.sst) /dev/shm/ToplingTemp/Topling-219654-Ynekmn 2024/11/20-13:11:21.827600 140257835828800 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029115.sst): first pass time = 0.27's, 195.355'MB/sec 2024/11/20-13:11:22.129685 140257835828800 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029115.sst): old prealloc_size = 147639500, real_size = 16119912 2024/11/20-13:11:22.129727 140257835828800 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029115.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 51684 prefix = 4 raw-key = 0.0004 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.0526 GB zip-val = 0.0161 GB avg-val =1016.97 avg-zval = 311.64 2024/11/20-13:11:22.191988 140257835828800 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00/029115.sst): second pass time = 0.30's, 174.358'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.06's, 267.248'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 8.399'MB/sec (index lex order gen) rebuild zvType time = 0.05's, 0.274'MB/sec write SST data time = 0.02's, 1100.115'MB/sec dict compress time = 0.00's, 529772.000'MB/sec z-dict build time = 0.00's, sample length = 0.530'MB, throughput = 13024.191'MB/sec zip my value time = 0.30's, unzip length = 0.053'GB zip my value throughput = 174.358'MB/sec zip pipeline throughput = 174.358'MB/sec entries = 51684 avg-key = 20.00 avg-zkey = 0.00 avg-val = 1016.97 avg-zval = 311.64 usrkeys = 51684 avg-key = 12.00 avg-zkey = 0.00 avg-val = 1016.97 avg-zval = 311.64 TagRS{ kinds = RS_Key0_TagN bytes = 8160 } TagArray{ size = 5 bytes = 32 } seq expand size = 35 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0006 GB value = 0.0526 GB dict = 0.53 MB all = 0.0536 GB } __ZipSize{ index = 0.0000 GB value = 0.0161 GB dict = 0.53 MB all = 0.0166 GB } UnZip/Zip{ index =3230.2500 value = 3.2633 dict = 1.00 all = 3.2197 } Zip/UnZip{ index = 0.0003 value = 0.3064 dict = 1.00 all = 0.3106 } ---------------------------- total value len = 2.685627 GB avg = 1.017 KB (by entry num) total key len = 0.052805 GB avg = 0.020 KB total ukey len = 0.031683 GB avg = 0.012 KB total ukey num = 0.002640246 Billion total entry num = 0.002640246 Billion write speed all = 0.173095514 MB/sec (with seq num) write speed all = 0.171760400 MB/sec (without seq num) 2024/11/20-13:11:22.218750 140257835828800 [compaction_job.cc:1948] [default] [JOB 6744] Generated table #29115: 51684 keys, 16645999 bytes, temperature: kWarm 2024/11/20-13:11:22.218811 140257835828800 EVENT_LOG_v1 {"time_micros": 1732079482218783, "cf_name": "default", "job": 6744, "event": "table_file_creation", "file_number": 29115, "file_size": 16645999, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 557088392, "table_properties": {"data_size": 16106608, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 1033680, "raw_average_key_size": 20, "raw_value_size": 52561170, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 51684, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1732079481, "oldest_key_time": 0, "file_creation_time": 1732079481, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 29115, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:11:22.233670 140257835828800 [compaction_job.cc:703] job-06744: subcompact[0], size: 0.851599 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06744/att-00 [029095,029096,029097,029098,029099,029100,029101,029102,029103,029104,029105,029106,029107,029108,029109,029110,029111,029112,029113,029114,029115] 2024/11/20-13:11:22.361694 140257835828800 (Original Log Time 2024/11/20-13:10:52.265288) EVENT_LOG_v1 {"time_micros": 1732079452265228, "job": 6744, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [29070], "files_L3": [29091, 29092, 28990, 29019, 29016, 29017, 29018, 29020, 29021, 29022, 29023, 29024, 29025, 29029, 29027, 29028, 29026, 29030, 29033, 29032, 29034, 29031], "score": 0.79189, "input_data_size": 864502945, "oldest_snapshot_seqno": 557489171} 2024/11/20-13:11:22.361696 140257835828800 (Original Log Time 2024/11/20-13:11:22.360106) [compaction_job.cc:2022] [default] [JOB 6744] Compacted 1@2 + 22@3 files to L3 => 851599164 bytes 2024/11/20-13:11:22.361698 140257835828800 (Original Log Time 2024/11/20-13:11:22.361618) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 28.8 rd, 28.4 wr, level 3, files in(1, 22) out(21 +0 blob) MB in(13.0, 811.5 +0.0 blob) out(812.1 +0.0 blob), read-write-amplify(125.9) write-amplify(62.5) OK, records in: 2652361, records dropped: 12115 output_compression: Snappy 2024/11/20-13:11:22.361701 140257835828800 (Original Log Time 2024/11/20-13:11:22.361669) EVENT_LOG_v1 {"time_micros": 1732079482361641, "cf": "default", "job": 6744, "event": "compaction_finished", "compaction_time_micros": 29968387, "compaction_time_cpu_micros": 7301464, "output_level": 3, "num_output_files": 21, "total_output_size": 851599164, "num_input_records": 2652361, "num_output_records": 2640246, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 587606350, "file_range_sync_nanos": 0, "file_fsync_nanos": 19881, "file_prepare_write_nanos": 45507, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/20-13:11:22.486165 140257835828800 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732076400,"starts":1732079452,"executesMs":30255,"compactionJobId":6744,"attempt":0,"compactionInputRawBytes":2750523684,"compactionInputZipBytes":864502945,"compactionOutputRawBytes":0,"compactionOutputZipBytes":851599164,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-f25c75ff33c118e592c05d4baead6fd3-63db31a3b0adc990-00"}}, response = 2024/11/20-13:11:22.486222 140257835828800 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-06744/att-00: olev 3, work 30.255 s, result 9.039 ms, install 1.608 ms, input{raw 2.562 GiB zip 824.454 MiB}