2024/11/20-21:18:20.669744 140008652125760 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00: fork to child time = 0.023978 sec 2024/11/20-21:18:20.670428 140008652125760 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00 2024/11/20-21:18:20.677928 140008652125760 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00 2024/11/20-21:18:20.680249 140008652125760 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-21:18:20.680559 140008652125760 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-21:18:20.727016 140008652125760 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-13722/att-00/MANIFEST-049091 2024/11/20-21:18:20.794978 140008652125760 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-13722/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 72232, last_sequence is 1151757336, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 72051 2024/11/20-21:18:20.794987 140008652125760 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-21:18:20.795114 140008652125760 [version_set.cc:5724] Creating manifest 72232 2024/11/20-21:18:20.827294 140008652125760 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-21:18:20.846412 140008652125760 [compaction_job.cc:2437] [default] [JOB 13722] Compacting 1@2 + 14@3 files to L3, score 0.92, subcompactions 1 : 1 2024/11/20-21:18:20.846432 140008652125760 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 11007 Base level 2, inputs: [72225(27MB)], [72168(40MB) 72169(36MB) 72170(37MB) 72171(37MB) 72172(36MB) 72173(37MB) 72174(36MB) 72175(40MB) 72176(37MB) 72177(38MB) 72178(38MB) 72179(38MB) 72180(38MB) 72181(38MB)] 2024/11/20-21:18:20.855071 140008652125760 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-21:18:20.855106 140008652125760 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-21:18:20.855134 140008652125760 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-21:18:20.855147 140008652125760 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-21:18:20.855161 140008652125760 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-21:18:21.462011 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072233.sst) /dev/shm/ToplingTemp/Topling-1002740-IatimG 2024/11/20-21:18:22.216245 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072233.sst): first pass time = 0.75's, 177.968'MB/sec 2024/11/20-21:18:23.596220 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072233.sst): old prealloc_size = 147639500, real_size = 38231056 2024/11/20-21:18:23.596991 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072233.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922986 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1084 GB zip-val = 0.0362 GB avg-val = 117.42 avg-zval = 39.23 2024/11/20-21:18:23.598673 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072233.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:24.741820 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072233.sst): second pass time = 1.38's, 78.590'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.15's, 36.945'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2414.469'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.272'MB/sec write SST data time = 0.30's, 142.655'MB/sec dict compress time = 6061892516.92's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31550.984'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.590'MB/sec zip pipeline throughput = 78.590'MB/sec entries = 922986 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.42 avg-zval = 39.23 usrkeys = 922986 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.42 avg-zval = 39.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922986 bytes = 3230464 } seq expand size = 6460902 multi value expand size = 0 cnt WriteAppend = 38 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0362 GB dict = 1.08 MB all = 0.0423 GB } UnZip/Zip{ index = 10.3085 value = 2.9930 dict = 1.00 all = 3.1713 } Zip/UnZip{ index = 0.0970 value = 0.3341 dict = 1.00 all = 0.3153 } ---------------------------- total value len = 563.180567 GB avg = 0.331 KB (by entry num) total key len = 44.340250 GB avg = 0.026 KB total ukey len = 32.812601 GB avg = 0.014 KB total ukey num = 2.321937562 Billion total entry num = 1.701439584 Billion write speed all = 52.003113113 MB/sec (with seq num) write speed all = 50.837982271 MB/sec (without seq num) 2024/11/20-21:18:24.796297 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72233: 922986 keys, 42322880 bytes, temperature: kWarm 2024/11/20-21:18:24.796423 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108704796365, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72233, "file_size": 42322880, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 939479839, "largest_seqno": 1150907764, "table_properties": {"data_size": 36209408, "index_size": 1790720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25843608, "raw_average_key_size": 28, "raw_value_size": 108374200, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922986, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108701, "oldest_key_time": 0, "file_creation_time": 1732108701, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72233, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:24.860237 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072234.sst) /dev/shm/ToplingTemp/Topling-1002740-zaSCgG 2024/11/20-21:18:25.913356 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072234.sst): first pass time = 1.05's, 127.455'MB/sec 2024/11/20-21:18:27.295354 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072234.sst): old prealloc_size = 147639500, real_size = 34150880 2024/11/20-21:18:27.295967 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072234.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923292 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0322 GB avg-val = 117.37 avg-zval = 34.84 2024/11/20-21:18:27.296761 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072234.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:28.495974 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072234.sst): second pass time = 1.38's, 78.454'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.20's, 31.851'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2967.109'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.245'MB/sec write SST data time = 0.26's, 148.778'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 19607.677'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.454'MB/sec zip pipeline throughput = 78.454'MB/sec entries = 923292 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.37 avg-zval = 34.84 usrkeys = 923292 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.37 avg-zval = 34.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923292 bytes = 3231536 } seq expand size = 6463044 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5134 value = 3.3692 dict = 1.00 all = 3.5099 } Zip/UnZip{ index = 0.0951 value = 0.2968 dict = 1.00 all = 0.2849 } ---------------------------- total value len = 563.295396 GB avg = 0.331 KB (by entry num) total key len = 44.366102 GB avg = 0.026 KB total ukey len = 32.831067 GB avg = 0.014 KB total ukey num = 2.322860854 Billion total entry num = 1.702362876 Billion write speed all = 51.998445365 MB/sec (with seq num) write speed all = 50.833056762 MB/sec (without seq num) 2024/11/20-21:18:28.558383 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72234: 923292 keys, 38240324 bytes, temperature: kWarm 2024/11/20-21:18:28.558425 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108708558403, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72234, "file_size": 38240324, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 952414140, "largest_seqno": 1150909122, "table_properties": {"data_size": 32163472, "index_size": 1756416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25852176, "raw_average_key_size": 28, "raw_value_size": 108365676, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923292, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108704, "oldest_key_time": 0, "file_creation_time": 1732108704, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72234, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:28.620993 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072235.sst) /dev/shm/ToplingTemp/Topling-1002740-VD8mGG 2024/11/20-21:18:29.430424 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072235.sst): first pass time = 0.81's, 165.825'MB/sec 2024/11/20-21:18:30.568066 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072235.sst): old prealloc_size = 147639500, real_size = 35458032 2024/11/20-21:18:30.568676 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072235.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927340 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 116.73 avg-zval = 36.07 2024/11/20-21:18:30.569460 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072235.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:31.783515 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072235.sst): second pass time = 1.14's, 95.223'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.22's, 32.452'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2995.038'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.243'MB/sec write SST data time = 0.26's, 151.132'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18407.246'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.223'MB/sec zip pipeline throughput = 95.223'MB/sec entries = 927340 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.73 avg-zval = 36.07 usrkeys = 927340 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.73 avg-zval = 36.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927340 bytes = 3129792 } seq expand size = 6491380 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4438 value = 3.2362 dict = 1.00 all = 3.4029 } Zip/UnZip{ index = 0.0958 value = 0.3090 dict = 1.00 all = 0.2939 } ---------------------------- total value len = 563.410140 GB avg = 0.331 KB (by entry num) total key len = 44.392068 GB avg = 0.026 KB total ukey len = 32.849614 GB avg = 0.014 KB total ukey num = 2.323788194 Billion total entry num = 1.703290216 Billion write speed all = 51.995858713 MB/sec (with seq num) write speed all = 50.830163211 MB/sec (without seq num) 2024/11/20-21:18:31.834850 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72235: 927340 keys, 39441954 bytes, temperature: kWarm 2024/11/20-21:18:31.834940 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108711834894, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72235, "file_size": 39441954, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1069697620, "largest_seqno": 1150908951, "table_properties": {"data_size": 33450144, "index_size": 1775872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965520, "raw_average_key_size": 28, "raw_value_size": 108252240, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927340, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108708, "oldest_key_time": 0, "file_creation_time": 1732108708, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72235, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:31.894681 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072236.sst) /dev/shm/ToplingTemp/Topling-1002740-D5MWVI 2024/11/20-21:18:32.956183 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072236.sst): first pass time = 1.06's, 126.447'MB/sec 2024/11/20-21:18:34.905866 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072236.sst): old prealloc_size = 147639500, real_size = 35001968 2024/11/20-21:18:34.906975 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072236.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923274 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0330 GB avg-val = 117.37 avg-zval = 35.76 2024/11/20-21:18:34.907857 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072236.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:36.265149 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072236.sst): second pass time = 1.95's, 55.608'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.36's, 28.759'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1614.620'MB/sec (index lex order gen) rebuild zvType time = 1.18's, 0.195'MB/sec write SST data time = 0.17's, 223.488'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 26105.847'MB/sec zip my value time = 1.95's, unzip length = 0.108'GB zip my value throughput = 55.608'MB/sec zip pipeline throughput = 240.272'MB/sec entries = 923274 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.37 avg-zval = 35.76 usrkeys = 923274 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.37 avg-zval = 35.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923274 bytes = 3231472 } seq expand size = 6462918 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5139 value = 3.2824 dict = 1.00 all = 3.4334 } Zip/UnZip{ index = 0.0951 value = 0.3047 dict = 1.00 all = 0.2913 } ---------------------------- total value len = 563.524969 GB avg = 0.331 KB (by entry num) total key len = 44.417919 GB avg = 0.026 KB total ukey len = 32.868080 GB avg = 0.014 KB total ukey num = 2.324711468 Billion total entry num = 1.704213490 Billion write speed all = 51.987961695 MB/sec (with seq num) write speed all = 50.822081316 MB/sec (without seq num) 2024/11/20-21:18:36.315215 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72236: 923274 keys, 39091700 bytes, temperature: kWarm 2024/11/20-21:18:36.315282 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108716315251, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72236, "file_size": 39091700, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 980655735, "largest_seqno": 1150908233, "table_properties": {"data_size": 33014688, "index_size": 1756288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25851672, "raw_average_key_size": 28, "raw_value_size": 108366184, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923274, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108711, "oldest_key_time": 0, "file_creation_time": 1732108711, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72236, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:36.357640 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072237.sst) /dev/shm/ToplingTemp/Topling-1002740-FIyNIG 2024/11/20-21:18:37.445245 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072237.sst): first pass time = 1.09's, 123.411'MB/sec 2024/11/20-21:18:39.754794 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072237.sst): old prealloc_size = 147639500, real_size = 33763904 2024/11/20-21:18:39.755361 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072237.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923942 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 117.27 avg-zval = 34.33 2024/11/20-21:18:39.756175 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072237.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:40.678998 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072237.sst): second pass time = 2.31's, 46.929'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.025'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3341.929'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.291'MB/sec write SST data time = 0.13's, 294.522'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 25737.503'MB/sec zip my value time = 2.31's, unzip length = 0.108'GB zip my value throughput = 46.929'MB/sec zip pipeline throughput = 153.477'MB/sec entries = 923942 avg-key = 28.00 avg-zkey = 1.97 avg-val = 117.27 avg-zval = 34.33 usrkeys = 923942 avg-key = 20.00 avg-zkey = 1.97 avg-val = 123.84 avg-zval = 34.33 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 867567 bytes = 3144944 } seq expand size = 6072969 multi value expand size = 0 cnt WriteAppend = 83 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.1749 value = 3.4161 dict = 1.00 all = 3.5400 } Zip/UnZip{ index = 0.0983 value = 0.2927 dict = 1.00 all = 0.2825 } ---------------------------- total value len = 563.639389 GB avg = 0.331 KB (by entry num) total key len = 44.443790 GB avg = 0.026 KB total ukey len = 32.886558 GB avg = 0.014 KB total ukey num = 2.325635410 Billion total entry num = 1.705137432 Billion write speed all = 51.980338705 MB/sec (with seq num) write speed all = 50.814266373 MB/sec (without seq num) 2024/11/20-21:18:40.726831 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72237: 923942 keys, 37914342 bytes, temperature: kWarm 2024/11/20-21:18:40.726897 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108720726865, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72237, "file_size": 37914342, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1150906816, "table_properties": {"data_size": 31716592, "index_size": 1816128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25870376, "raw_average_key_size": 28, "raw_value_size": 108347480, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923942, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108716, "oldest_key_time": 0, "file_creation_time": 1732108716, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72237, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:40.764982 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072238.sst) /dev/shm/ToplingTemp/Topling-1002740-W7TJeF 2024/11/20-21:18:41.547957 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072238.sst): first pass time = 0.78's, 171.428'MB/sec 2024/11/20-21:18:43.217786 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072238.sst): old prealloc_size = 147639500, real_size = 36396096 2024/11/20-21:18:43.218428 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072238.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926406 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0343 GB avg-val = 116.88 avg-zval = 37.07 2024/11/20-21:18:43.218990 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072238.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:43.976752 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072238.sst): second pass time = 1.67's, 64.878'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 52.024'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2933.369'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.365'MB/sec write SST data time = 0.12's, 317.786'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 17038.162'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 64.878'MB/sec zip pipeline throughput = 64.878'MB/sec entries = 926406 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.88 avg-zval = 37.07 usrkeys = 926406 avg-key = 20.00 avg-zkey = 1.97 avg-val = 121.55 avg-zval = 37.07 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 617462 bytes = 2083952 } seq expand size = 4322234 multi value expand size = 0 cnt WriteAppend = 51 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.1765 value = 3.1528 dict = 1.00 all = 3.3994 } Zip/UnZip{ index = 0.0983 value = 0.3172 dict = 1.00 all = 0.2942 } ---------------------------- total value len = 563.751990 GB avg = 0.330 KB (by entry num) total key len = 44.469729 GB avg = 0.026 KB total ukey len = 32.905087 GB avg = 0.014 KB total ukey num = 2.326561816 Billion total entry num = 1.706063838 Billion write speed all = 51.977528989 MB/sec (with seq num) write speed all = 50.811151928 MB/sec (without seq num) 2024/11/20-21:18:44.023237 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72238: 926406 keys, 39483290 bytes, temperature: kWarm 2024/11/20-21:18:44.023302 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108724023273, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72238, "file_size": 39483290, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 978567417, "table_properties": {"data_size": 34343664, "index_size": 1820672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939368, "raw_average_key_size": 28, "raw_value_size": 108278400, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926406, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108720, "oldest_key_time": 0, "file_creation_time": 1732108720, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72238, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:44.054800 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072239.sst) /dev/shm/ToplingTemp/Topling-1002740-AJ2nVG 2024/11/20-21:18:44.836963 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072239.sst): first pass time = 0.78's, 171.606'MB/sec 2024/11/20-21:18:46.162708 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072239.sst): old prealloc_size = 147639500, real_size = 36455952 2024/11/20-21:18:46.163402 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072239.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926896 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.80 avg-zval = 37.12 2024/11/20-21:18:46.163606 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072239.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:46.916874 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072239.sst): second pass time = 1.32's, 81.718'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.75's, 49.916'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2729.393'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.367'MB/sec write SST data time = 0.12's, 306.496'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33940.426'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.718'MB/sec zip pipeline throughput = 81.718'MB/sec entries = 926896 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.80 avg-zval = 37.12 usrkeys = 926896 avg-key = 20.00 avg-zkey = 1.97 avg-val = 117.22 avg-zval = 37.12 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 55059 bytes = 185840 } seq expand size = 385413 multi value expand size = 0 cnt WriteAppend = 39 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.1748 value = 3.1470 dict = 1.00 all = 3.5654 } Zip/UnZip{ index = 0.0983 value = 0.3178 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 563.860640 GB avg = 0.330 KB (by entry num) total key len = 44.495682 GB avg = 0.026 KB total ukey len = 32.923624 GB avg = 0.014 KB total ukey num = 2.327488712 Billion total entry num = 1.706990734 Billion write speed all = 51.975972575 MB/sec (with seq num) write speed all = 50.809254974 MB/sec (without seq num) 2024/11/20-21:18:46.963760 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72239: 926896 keys, 37644811 bytes, temperature: kWarm 2024/11/20-21:18:46.963831 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108726963799, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72239, "file_size": 37644811, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 972340122, "table_properties": {"data_size": 34402112, "index_size": 1821952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953088, "raw_average_key_size": 28, "raw_value_size": 108264660, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926896, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108724, "oldest_key_time": 0, "file_creation_time": 1732108724, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72239, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:46.997958 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072240.sst) /dev/shm/ToplingTemp/Topling-1002740-STuifH 2024/11/20-21:18:47.814190 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072240.sst): first pass time = 0.82's, 164.444'MB/sec 2024/11/20-21:18:49.114660 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072240.sst): old prealloc_size = 147639500, real_size = 38575616 2024/11/20-21:18:49.115222 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072240.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926105 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0365 GB avg-val = 116.93 avg-zval = 39.46 2024/11/20-21:18:49.115872 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072240.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:49.900962 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072240.sst): second pass time = 1.30's, 83.328'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 53.408'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3435.361'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.351'MB/sec write SST data time = 0.13's, 331.890'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 17057.150'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.328'MB/sec zip pipeline throughput = 83.328'MB/sec entries = 926105 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.93 avg-zval = 39.46 usrkeys = 926105 avg-key = 20.00 avg-zkey = 1.94 avg-val = 122.15 avg-zval = 39.46 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 690393 bytes = 2416384 } seq expand size = 4832751 multi value expand size = 0 cnt WriteAppend = 22 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0365 GB dict = 1.08 MB all = 0.0420 GB } UnZip/Zip{ index = 10.3124 value = 2.9629 dict = 1.00 all = 3.1960 } Zip/UnZip{ index = 0.0970 value = 0.3375 dict = 1.00 all = 0.3129 } ---------------------------- total value len = 563.973760 GB avg = 0.330 KB (by entry num) total key len = 44.521613 GB avg = 0.026 KB total ukey len = 32.942147 GB avg = 0.014 KB total ukey num = 2.328414817 Billion total entry num = 1.707916839 Billion write speed all = 51.974601648 MB/sec (with seq num) write speed all = 50.807548601 MB/sec (without seq num) 2024/11/20-21:18:49.957172 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72240: 926105 keys, 41995266 bytes, temperature: kWarm 2024/11/20-21:18:49.957240 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108729957209, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72240, "file_size": 41995266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1150909267, "table_properties": {"data_size": 36547824, "index_size": 1796096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25930940, "raw_average_key_size": 28, "raw_value_size": 108286858, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926105, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108726, "oldest_key_time": 0, "file_creation_time": 1732108726, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72240, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:49.997730 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072241.sst) /dev/shm/ToplingTemp/Topling-1002740-DVmLpG 2024/11/20-21:18:50.807746 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072241.sst): first pass time = 0.81's, 165.707'MB/sec 2024/11/20-21:18:51.980278 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072241.sst): old prealloc_size = 147639500, real_size = 36375504 2024/11/20-21:18:51.980942 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072241.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923865 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 117.28 avg-zval = 37.21 2024/11/20-21:18:51.981537 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072241.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:52.822539 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072241.sst): second pass time = 1.17's, 92.475'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.927'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2823.421'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 314.800'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34173.122'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.475'MB/sec zip pipeline throughput = 92.475'MB/sec entries = 923865 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.28 avg-zval = 37.21 usrkeys = 923865 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.92 avg-zval = 37.21 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 612498 bytes = 2143760 } seq expand size = 4287486 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4445 value = 3.1520 dict = 1.00 all = 3.3958 } Zip/UnZip{ index = 0.0957 value = 0.3173 dict = 1.00 all = 0.2945 } ---------------------------- total value len = 564.086397 GB avg = 0.330 KB (by entry num) total key len = 44.547481 GB avg = 0.026 KB total ukey len = 32.960624 GB avg = 0.014 KB total ukey num = 2.329338682 Billion total entry num = 1.708840704 Billion write speed all = 51.973462272 MB/sec (with seq num) write speed all = 50.806069249 MB/sec (without seq num) 2024/11/20-21:18:52.870457 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72241: 923865 keys, 39524822 bytes, temperature: kWarm 2024/11/20-21:18:52.870527 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108732870496, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72241, "file_size": 39524822, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1150909283, "table_properties": {"data_size": 34375296, "index_size": 1769088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868220, "raw_average_key_size": 28, "raw_value_size": 108349618, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923865, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108729, "oldest_key_time": 0, "file_creation_time": 1732108729, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72241, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:52.903083 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072242.sst) /dev/shm/ToplingTemp/Topling-1002740-OhAQiJ 2024/11/20-21:18:53.727357 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072242.sst): first pass time = 0.82's, 162.841'MB/sec 2024/11/20-21:18:54.863491 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072242.sst): old prealloc_size = 147639500, real_size = 35877136 2024/11/20-21:18:54.864157 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072242.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922450 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0339 GB avg-val = 117.50 avg-zval = 36.73 2024/11/20-21:18:54.864944 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072242.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:55.768521 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072242.sst): second pass time = 1.14's, 95.482'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 44.118'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2810.851'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.13's, 309.655'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33847.274'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.482'MB/sec zip pipeline throughput = 95.482'MB/sec entries = 922450 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.50 avg-zval = 36.73 usrkeys = 922450 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.11 avg-zval = 36.73 TagRS{ kinds = RS_Key0_TagN bytes = 144240 } TagArray{ size = 870848 bytes = 3047984 } seq expand size = 6095936 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.4433 value = 3.1992 dict = 1.00 all = 3.3615 } Zip/UnZip{ index = 0.0958 value = 0.3126 dict = 1.00 all = 0.2975 } ---------------------------- total value len = 564.200882 GB avg = 0.330 KB (by entry num) total key len = 44.573310 GB avg = 0.026 KB total ukey len = 32.979073 GB avg = 0.014 KB total ukey num = 2.330261132 Billion total entry num = 1.709763154 Billion write speed all = 51.972369589 MB/sec (with seq num) write speed all = 50.804640159 MB/sec (without seq num) 2024/11/20-21:18:55.816816 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72242: 922450 keys, 39927842 bytes, temperature: kWarm 2024/11/20-21:18:55.816885 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108735816853, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72242, "file_size": 39927842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1150908981, "table_properties": {"data_size": 33879744, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25828600, "raw_average_key_size": 28, "raw_value_size": 108389204, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922450, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108732, "oldest_key_time": 0, "file_creation_time": 1732108732, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72242, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:55.856119 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072243.sst) /dev/shm/ToplingTemp/Topling-1002740-7ju23F 2024/11/20-21:18:56.682355 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072243.sst): first pass time = 0.83's, 162.454'MB/sec 2024/11/20-21:18:57.826318 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072243.sst): old prealloc_size = 147639500, real_size = 35813328 2024/11/20-21:18:57.826970 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072243.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923521 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0338 GB avg-val = 117.33 avg-zval = 36.63 2024/11/20-21:18:57.827748 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072243.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:18:58.660346 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072243.sst): second pass time = 1.14's, 94.797'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.706'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2839.861'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 306.460'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 21492.526'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 94.797'MB/sec zip pipeline throughput = 94.797'MB/sec entries = 923521 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 36.63 usrkeys = 923521 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 36.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923521 bytes = 3116896 } seq expand size = 6464647 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.5114 value = 3.2035 dict = 1.00 all = 3.3733 } Zip/UnZip{ index = 0.0951 value = 0.3122 dict = 1.00 all = 0.2964 } ---------------------------- total value len = 564.315706 GB avg = 0.330 KB (by entry num) total key len = 44.599168 GB avg = 0.026 KB total ukey len = 32.997543 GB avg = 0.014 KB total ukey num = 2.331184653 Billion total entry num = 1.710686675 Billion write speed all = 51.971549124 MB/sec (with seq num) write speed all = 50.803477325 MB/sec (without seq num) 2024/11/20-21:18:58.710738 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72243: 923521 keys, 39787854 bytes, temperature: kWarm 2024/11/20-21:18:58.710810 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108738710778, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72243, "file_size": 39787854, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1063676601, "largest_seqno": 1150908731, "table_properties": {"data_size": 33825088, "index_size": 1757184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25858588, "raw_average_key_size": 28, "raw_value_size": 108359218, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923521, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108735, "oldest_key_time": 0, "file_creation_time": 1732108735, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72243, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:18:58.747262 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072244.sst) /dev/shm/ToplingTemp/Topling-1002740-lCW9TF 2024/11/20-21:18:59.574086 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072244.sst): first pass time = 0.83's, 162.338'MB/sec 2024/11/20-21:19:00.687289 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072244.sst): old prealloc_size = 147639500, real_size = 36084288 2024/11/20-21:19:00.687831 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072244.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922432 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0341 GB avg-val = 117.50 avg-zval = 36.97 2024/11/20-21:19:00.688660 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072244.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:19:01.545132 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072244.sst): second pass time = 1.11's, 97.438'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 46.699'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3388.839'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 307.223'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33597.457'MB/sec zip my value time = 1.11's, unzip length = 0.108'GB zip my value throughput = 97.438'MB/sec zip pipeline throughput = 97.438'MB/sec entries = 922432 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.50 avg-zval = 36.97 usrkeys = 922432 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.50 avg-zval = 36.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922432 bytes = 3113216 } seq expand size = 6457024 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.5128 value = 3.1787 dict = 1.00 all = 3.3504 } Zip/UnZip{ index = 0.0951 value = 0.3146 dict = 1.00 all = 0.2985 } ---------------------------- total value len = 564.430552 GB avg = 0.330 KB (by entry num) total key len = 44.624997 GB avg = 0.026 KB total ukey len = 33.015992 GB avg = 0.014 KB total ukey num = 2.332107085 Billion total entry num = 1.711609107 Billion write speed all = 51.970759628 MB/sec (with seq num) write speed all = 50.802345672 MB/sec (without seq num) 2024/11/20-21:19:01.593521 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72244: 922432 keys, 40059918 bytes, temperature: kWarm 2024/11/20-21:19:01.593598 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108741593556, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72244, "file_size": 40059918, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1063678836, "largest_seqno": 1150908747, "table_properties": {"data_size": 34098608, "index_size": 1754880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25828096, "raw_average_key_size": 28, "raw_value_size": 108389704, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108738, "oldest_key_time": 0, "file_creation_time": 1732108738, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72244, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:19:01.637038 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072245.sst) /dev/shm/ToplingTemp/Topling-1002740-PhDASF 2024/11/20-21:19:02.506906 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072245.sst): first pass time = 0.87's, 154.307'MB/sec 2024/11/20-21:19:03.827376 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072245.sst): old prealloc_size = 147639500, real_size = 36010336 2024/11/20-21:19:03.827949 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072245.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923565 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0340 GB avg-val = 117.33 avg-zval = 36.84 2024/11/20-21:19:03.828723 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072245.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:19:04.671133 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072245.sst): second pass time = 1.32's, 82.112'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.390'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.606'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 308.760'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30518.915'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.112'MB/sec zip pipeline throughput = 82.112'MB/sec entries = 923565 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 36.84 usrkeys = 923565 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 36.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923565 bytes = 3117040 } seq expand size = 6464955 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.5134 value = 3.1849 dict = 1.00 all = 3.3567 } Zip/UnZip{ index = 0.0951 value = 0.3140 dict = 1.00 all = 0.2979 } ---------------------------- total value len = 564.545375 GB avg = 0.330 KB (by entry num) total key len = 44.650856 GB avg = 0.026 KB total ukey len = 33.034463 GB avg = 0.014 KB total ukey num = 2.333030650 Billion total entry num = 1.712532672 Billion write speed all = 51.968901858 MB/sec (with seq num) write speed all = 50.800169187 MB/sec (without seq num) 2024/11/20-21:19:04.728176 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72245: 923565 keys, 39984916 bytes, temperature: kWarm 2024/11/20-21:19:04.728269 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108744728221, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72245, "file_size": 39984916, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1063678877, "largest_seqno": 1150908454, "table_properties": {"data_size": 34022352, "index_size": 1756928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859820, "raw_average_key_size": 28, "raw_value_size": 108357938, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923565, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108741, "oldest_key_time": 0, "file_creation_time": 1732108741, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72245, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:19:04.806479 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072246.sst) /dev/shm/ToplingTemp/Topling-1002740-2TEn6G 2024/11/20-21:19:05.825151 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072246.sst): first pass time = 1.02's, 131.765'MB/sec 2024/11/20-21:19:07.512399 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072246.sst): old prealloc_size = 147639500, real_size = 36246416 2024/11/20-21:19:07.513126 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072246.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923648 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0342 GB avg-val = 117.31 avg-zval = 37.08 2024/11/20-21:19:07.514004 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072246.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-21:19:08.574106 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072246.sst): second pass time = 1.69's, 64.257'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 38.120'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2518.632'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.251'MB/sec write SST data time = 0.14's, 287.254'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 28213.686'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.257'MB/sec zip pipeline throughput = 126.813'MB/sec entries = 923648 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.31 avg-zval = 37.08 usrkeys = 923648 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.28 avg-zval = 37.08 TagRS{ kinds = RS_Key0_TagN bytes = 144400 } TagArray{ size = 919972 bytes = 3219920 } seq expand size = 6439804 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.4425 value = 3.1640 dict = 1.00 all = 3.3163 } Zip/UnZip{ index = 0.0958 value = 0.3161 dict = 1.00 all = 0.3015 } ---------------------------- total value len = 564.660171 GB avg = 0.330 KB (by entry num) total key len = 44.676719 GB avg = 0.026 KB total ukey len = 33.052936 GB avg = 0.014 KB total ukey num = 2.333954298 Billion total entry num = 1.713456320 Billion write speed all = 51.963599585 MB/sec (with seq num) write speed all = 50.794625775 MB/sec (without seq num) 2024/11/20-21:19:08.628674 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72246: 923648 keys, 40471872 bytes, temperature: kWarm 2024/11/20-21:19:08.628744 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108748628710, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72246, "file_size": 40471872, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1150907592, "table_properties": {"data_size": 34246272, "index_size": 1769024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25862144, "raw_average_key_size": 28, "raw_value_size": 108355628, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923648, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108744, "oldest_key_time": 0, "file_creation_time": 1732108744, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72246, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:19:08.668948 140008652125760 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072247.sst) /dev/shm/ToplingTemp/Topling-1002740-h7PfpJ 2024/11/20-21:19:08.739771 140008652125760 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072247.sst): first pass time = 0.07's, 249.808'MB/sec 2024/11/20-21:19:08.892502 140008652125760 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072247.sst): old prealloc_size = 147639500, real_size = 5298256 2024/11/20-21:19:08.892646 140008652125760 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072247.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 121107 prefix = 4 raw-key = 0.0019 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.0143 GB zip-val = 0.0050 GB avg-val = 118.00 avg-zval = 41.56 2024/11/20-21:19:08.892750 140008652125760 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072247.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/20-21:19:09.001275 140008652125760 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00/072247.sst): second pass time = 0.15's, 93.993'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.11's, 51.700'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2123.972'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.334'MB/sec write SST data time = 0.02's, 314.145'MB/sec dict compress time = -4257855122.03's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 0.149'MB, throughput = 5837.016'MB/sec zip my value time = 0.15's, unzip length = 0.014'GB zip my value throughput = 93.993'MB/sec zip pipeline throughput = 93.993'MB/sec entries = 121107 avg-key = 28.00 avg-zkey = 1.94 avg-val = 118.00 avg-zval = 41.56 usrkeys = 121107 avg-key = 20.00 avg-zkey = 1.94 avg-val = 121.18 avg-zval = 41.56 TagRS{ kinds = RS_Key0_TagN bytes = 19040 } TagArray{ size = 55012 bytes = 185680 } seq expand size = 385084 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0024 GB value = 0.0143 GB dict = 0.15 MB all = 0.0177 GB } __ZipSize{ index = 0.0002 GB value = 0.0050 GB dict = 0.15 MB all = 0.0056 GB } UnZip/Zip{ index = 10.3179 value = 2.8394 dict = 1.00 all = 3.1443 } Zip/UnZip{ index = 0.0969 value = 0.3522 dict = 1.00 all = 0.3180 } ---------------------------- total value len = 564.674846 GB avg = 0.330 KB (by entry num) total key len = 44.680110 GB avg = 0.026 KB total ukey len = 33.055358 GB avg = 0.014 KB total ukey num = 2.334075405 Billion total entry num = 1.713577427 Billion write speed all = 51.963247347 MB/sec (with seq num) write speed all = 50.794233500 MB/sec (without seq num) 2024/11/20-21:19:09.015706 140008652125760 [compaction_job.cc:1948] [default] [JOB 13722] Generated table #72247: 121107 keys, 5623349 bytes, temperature: kWarm 2024/11/20-21:19:09.015761 140008652125760 EVENT_LOG_v1 {"time_micros": 1732108749015731, "cf_name": "default", "job": 13722, "event": "table_file_creation", "file_number": 72247, "file_size": 5623349, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1140654301, "table_properties": {"data_size": 5033024, "index_size": 234752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3390996, "raw_average_key_size": 28, "raw_value_size": 14290626, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 121107, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732108748, "oldest_key_time": 0, "file_creation_time": 1732108748, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 72247, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-21:19:09.031085 140008652125760 [compaction_job.cc:703] job-13722: subcompact[0], size: 0.561515 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13722/att-00 [072233,072234,072235,072236,072237,072238,072239,072240,072241,072242,072243,072244,072245,072246,072247] 2024/11/20-21:19:09.155864 140008652125760 (Original Log Time 2024/11/20-21:18:20.846482) EVENT_LOG_v1 {"time_micros": 1732108700846445, "job": 13722, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [72225], "files_L3": [72168, 72169, 72170, 72171, 72172, 72173, 72174, 72175, 72176, 72177, 72178, 72179, 72180, 72181], "score": 0.915509, "input_data_size": 585079331, "oldest_snapshot_seqno": 1151756616} 2024/11/20-21:19:09.155868 140008652125760 (Original Log Time 2024/11/20-21:19:09.144158) [compaction_job.cc:2022] [default] [JOB 13722] Compacted 1@2 + 14@3 files to L3 => 561515140 bytes 2024/11/20-21:19:09.155869 140008652125760 (Original Log Time 2024/11/20-21:19:09.155795) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 12.1 rd, 11.7 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.1, 530.9 +0.0 blob) out(535.5 +0.0 blob), read-write-amplify(40.3) write-amplify(19.8) OK, records in: 13172330, records dropped: 111501 output_compression: Snappy 2024/11/20-21:19:09.155873 140008652125760 (Original Log Time 2024/11/20-21:19:09.155842) EVENT_LOG_v1 {"time_micros": 1732108749155817, "cf": "default", "job": 13722, "event": "compaction_finished", "compaction_time_micros": 48184609, "compaction_time_cpu_micros": 21497942, "output_level": 3, "num_output_files": 15, "total_output_size": 561515140, "num_input_records": 13172330, "num_output_records": 13060829, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 440794519, "file_range_sync_nanos": 0, "file_fsync_nanos": 13873, "file_prepare_write_nanos": 48032, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/20-21:19:09.291146 140008652125760 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732108700,"executesMs":48468,"compactionJobId":13722,"attempt":0,"compactionInputRawBytes":1912563988,"compactionInputZipBytes":585079331,"compactionOutputRawBytes":0,"compactionOutputZipBytes":561515140,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-d78eb7c2a28c9286bb5cde6d6411b570-9b42ac097b422f3c-00"}}, response = 2024/11/20-21:19:09.291213 140008652125760 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-13722/att-00: olev 3, work 48.468 s, result 5.954 ms, install 11.732 ms, input{raw 1.781 GiB zip 557.975 MiB}