2024/11/22-18:54:13.157679 140025104320064 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00: fork to child time = 0.009199 sec 2024/11/22-18:54:13.158499 140025104320064 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00 2024/11/22-18:54:13.165027 140025104320064 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00 2024/11/22-18:54:13.166150 140025104320064 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-18:54:13.166404 140025104320064 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-18:54:13.212857 140025104320064 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08108/att-00/MANIFEST-206347 2024/11/22-18:54:13.241084 140025104320064 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08108/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 221705, last_sequence is 3481759267, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 221495 2024/11/22-18:54:13.241092 140025104320064 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-18:54:13.241481 140025104320064 [version_set.cc:5721] Creating manifest 221705 2024/11/22-18:54:13.274619 140025104320064 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-18:54:13.295964 140025104320064 [compaction_job.cc:2437] [default] [JOB 8108] Compacting 1@2 + 17@3 files to L3, score 0.97, subcompactions 1 : 1 2024/11/22-18:54:13.295983 140025104320064 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 6155 Base level 2, inputs: [221643(27MB)], [221700(35MB) 221701(5759KB) 221650(447KB) 221599(35MB) 221600(35MB) 221601(35MB) 221602(35MB) 221603(35MB) 221604(35MB) 221605(33MB) 221606(8411KB) 221573(35MB) 221574(35MB) 221575(35MB) 221576(35MB) 221577(34MB) 221578(36MB)] 2024/11/22-18:54:13.301863 140025104320064 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:54:13.301894 140025104320064 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:54:13.301914 140025104320064 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:54:13.301919 140025104320064 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-18:54:13.301930 140025104320064 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-18:54:13.854528 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221706.sst) /dev/shm/ToplingTemp/Topling-248533-TaXjd5 2024/11/22-18:54:14.398714 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221706.sst): first pass time = 0.54's, 246.661'MB/sec 2024/11/22-18:54:15.734843 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221706.sst): old prealloc_size = 147639500, real_size = 33480576 2024/11/22-18:54:15.735380 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221706.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926400 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.88 avg-zval = 33.99 2024/11/22-18:54:15.736138 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221706.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:16.652205 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221706.sst): second pass time = 1.34's, 81.086'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.836'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3417.893'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 294.608'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29016.333'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.086'MB/sec zip pipeline throughput = 81.086'MB/sec entries = 926400 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.88 avg-zval = 33.99 usrkeys = 926400 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.88 avg-zval = 33.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926400 bytes = 3126608 } seq expand size = 6484800 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5147 value = 3.4389 dict = 1.00 all = 3.5828 } Zip/UnZip{ index = 0.0951 value = 0.2908 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 0.114763 GB avg = 0.124 KB (by entry num) total key len = 0.025939 GB avg = 0.028 KB total ukey len = 0.018528 GB avg = 0.020 KB total ukey num = 0.000926400 Billion total entry num = 0.000926400 Billion write speed all = 0.003896115 MB/sec (with seq num) write speed all = 0.003690896 MB/sec (without seq num) 2024/11/22-18:54:16.686466 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221706: 926400 keys, 37461618 bytes, temperature: kWarm 2024/11/22-18:54:16.686561 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272856686501, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221706, "file_size": 37461618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774735, "largest_seqno": 3480996692, "table_properties": {"data_size": 31486704, "index_size": 1762112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939200, "raw_average_key_size": 28, "raw_value_size": 108278568, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926400, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272853, "oldest_key_time": 0, "file_creation_time": 1732272853, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221706, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:16.721525 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221707.sst) /dev/shm/ToplingTemp/Topling-248533-V82JP5 2024/11/22-18:54:17.617664 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221707.sst): first pass time = 0.90's, 149.781'MB/sec 2024/11/22-18:54:18.938513 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221707.sst): old prealloc_size = 147639500, real_size = 33438128 2024/11/22-18:54:18.939066 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221707.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922861 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0314 GB avg-val = 117.44 avg-zval = 34.07 2024/11/22-18:54:18.939801 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221707.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:19.865849 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221707.sst): second pass time = 1.32's, 82.097'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.343'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3323.644'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 297.727'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32178.264'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.097'MB/sec zip pipeline throughput = 82.097'MB/sec entries = 922861 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.44 avg-zval = 34.07 usrkeys = 922861 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.44 avg-zval = 34.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922861 bytes = 3114672 } seq expand size = 6460027 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4445 value = 3.4471 dict = 1.00 all = 3.5877 } Zip/UnZip{ index = 0.0957 value = 0.2901 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 0.229601 GB avg = 0.124 KB (by entry num) total key len = 0.051779 GB avg = 0.028 KB total ukey len = 0.036985 GB avg = 0.020 KB total ukey num = 0.001849261 Billion total entry num = 0.001849261 Billion write speed all = 0.007790851 MB/sec (with seq num) write speed all = 0.007381233 MB/sec (without seq num) 2024/11/22-18:54:19.900748 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221707: 922861 keys, 37410944 bytes, temperature: kWarm 2024/11/22-18:54:19.900819 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272859900785, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221707, "file_size": 37410944, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774766, "largest_seqno": 3480996712, "table_properties": {"data_size": 31440032, "index_size": 1767168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25840108, "raw_average_key_size": 28, "raw_value_size": 108377686, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922861, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272856, "oldest_key_time": 0, "file_creation_time": 1732272856, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221707, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:19.942363 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221708.sst) /dev/shm/ToplingTemp/Topling-248533-rXfkH4 2024/11/22-18:54:20.757805 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221708.sst): first pass time = 0.82's, 164.603'MB/sec 2024/11/22-18:54:22.031214 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221708.sst): old prealloc_size = 147639500, real_size = 33215936 2024/11/22-18:54:22.031760 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221708.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926008 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.94 avg-zval = 33.72 2024/11/22-18:54:22.032481 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221708.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:22.968328 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221708.sst): second pass time = 1.27's, 85.087'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.693'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3375.205'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 293.940'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30018.499'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.087'MB/sec zip pipeline throughput = 85.087'MB/sec entries = 926008 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 33.72 usrkeys = 926008 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.94 avg-zval = 33.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926008 bytes = 3125296 } seq expand size = 6482056 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5140 value = 3.4683 dict = 1.00 all = 3.6084 } Zip/UnZip{ index = 0.0951 value = 0.2883 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 0.344373 GB avg = 0.124 KB (by entry num) total key len = 0.077708 GB avg = 0.028 KB total ukey len = 0.055505 GB avg = 0.020 KB total ukey num = 0.002775269 Billion total entry num = 0.002775269 Billion write speed all = 0.011685539 MB/sec (with seq num) write speed all = 0.011070859 MB/sec (without seq num) 2024/11/22-18:54:23.003982 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221708: 926008 keys, 37196232 bytes, temperature: kWarm 2024/11/22-18:54:23.004047 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272863004014, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221708, "file_size": 37196232, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774786, "largest_seqno": 3480996737, "table_properties": {"data_size": 31222768, "index_size": 1761472, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928224, "raw_average_key_size": 28, "raw_value_size": 108289512, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272859, "oldest_key_time": 0, "file_creation_time": 1732272859, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221708, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:23.039841 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221709.sst) /dev/shm/ToplingTemp/Topling-248533-xR6Wu4 2024/11/22-18:54:23.874337 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221709.sst): first pass time = 0.83's, 160.845'MB/sec 2024/11/22-18:54:25.192224 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221709.sst): old prealloc_size = 147639500, real_size = 33821504 2024/11/22-18:54:25.192757 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221709.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925734 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0318 GB avg-val = 116.99 avg-zval = 34.38 2024/11/22-18:54:25.193354 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221709.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:26.057987 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221709.sst): second pass time = 1.32's, 82.248'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.819'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3434.904'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 286.381'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32177.355'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.248'MB/sec zip pipeline throughput = 82.248'MB/sec entries = 925734 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.99 avg-zval = 34.38 usrkeys = 925734 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.02 avg-zval = 34.38 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 665480 bytes = 2246016 } seq expand size = 4658360 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5136 value = 3.4025 dict = 1.00 all = 3.6206 } Zip/UnZip{ index = 0.0951 value = 0.2939 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 0.457328 GB avg = 0.124 KB (by entry num) total key len = 0.103628 GB avg = 0.028 KB total ukey len = 0.074020 GB avg = 0.020 KB total ukey num = 0.003701003 Billion total entry num = 0.003701003 Billion write speed all = 0.015529078 MB/sec (with seq num) write speed all = 0.014709433 MB/sec (without seq num) 2024/11/22-18:54:26.091093 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221709: 925734 keys, 37070706 bytes, temperature: kWarm 2024/11/22-18:54:26.091160 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272866091125, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221709, "file_size": 37070706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3480996750, "table_properties": {"data_size": 31828848, "index_size": 1761024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920552, "raw_average_key_size": 28, "raw_value_size": 108297216, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925734, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272863, "oldest_key_time": 0, "file_creation_time": 1732272863, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221709, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:26.124907 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221710.sst) /dev/shm/ToplingTemp/Topling-248533-0ccQx4 2024/11/22-18:54:27.003000 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221710.sst): first pass time = 0.88's, 152.858'MB/sec 2024/11/22-18:54:28.445300 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221710.sst): old prealloc_size = 147639500, real_size = 34921440 2024/11/22-18:54:28.445960 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221710.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081414 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.11 avg-zval = 30.14 2024/11/22-18:54:28.445997 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221710.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:29.071158 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221710.sst): second pass time = 1.44's, 72.098'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.039'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3217.422'MB/sec (index lex order gen) rebuild zvType time = 0.48's, 0.558'MB/sec write SST data time = 0.14's, 253.433'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.035'MB, throughput = 29424.800'MB/sec zip my value time = 1.44's, unzip length = 0.104'GB zip my value throughput = 72.098'MB/sec zip pipeline throughput = 72.098'MB/sec entries = 1081414 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.11 avg-zval = 30.14 usrkeys = 1081414 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.11 avg-zval = 30.14 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.04 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5170 value = 3.1888 dict = 1.00 all = 3.7599 } Zip/UnZip{ index = 0.0951 value = 0.3136 dict = 1.00 all = 0.2660 } ---------------------------- total value len = 0.561266 GB avg = 0.117 KB (by entry num) total key len = 0.133908 GB avg = 0.028 KB total ukey len = 0.095648 GB avg = 0.020 KB total ukey num = 0.004782417 Billion total entry num = 0.004782417 Billion write speed all = 0.019243053 MB/sec (with seq num) write speed all = 0.018183999 MB/sec (without seq num) 2024/11/22-18:54:29.104391 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221710: 1081414 keys, 35697359 bytes, temperature: kWarm 2024/11/22-18:54:29.104456 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272869104422, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221710, "file_size": 35697359, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32594384, "index_size": 2056512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30279592, "raw_average_key_size": 28, "raw_value_size": 103938141, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081414, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272866, "oldest_key_time": 0, "file_creation_time": 1732272866, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221710, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:29.141124 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221711.sst) /dev/shm/ToplingTemp/Topling-248533-8csv17 2024/11/22-18:54:30.065552 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221711.sst): first pass time = 0.92's, 145.197'MB/sec 2024/11/22-18:54:31.390926 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221711.sst): old prealloc_size = 147639500, real_size = 33777504 2024/11/22-18:54:31.391434 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221711.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923294 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.37 avg-zval = 34.43 2024/11/22-18:54:31.392190 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221711.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:32.297537 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221711.sst): second pass time = 1.32's, 81.806'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.751'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3623.189'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.298'MB/sec write SST data time = 0.13's, 287.880'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 28030.545'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.806'MB/sec zip pipeline throughput = 81.806'MB/sec entries = 923294 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.37 avg-zval = 34.43 usrkeys = 923294 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.01 avg-zval = 34.43 TagRS{ kinds = RS_Key0_TagN bytes = 144360 } TagArray{ size = 876559 bytes = 3067968 } seq expand size = 6135913 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5115 value = 3.4088 dict = 1.00 all = 3.5459 } Zip/UnZip{ index = 0.0951 value = 0.2934 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 0.675768 GB avg = 0.118 KB (by entry num) total key len = 0.159760 GB avg = 0.028 KB total ukey len = 0.114114 GB avg = 0.020 KB total ukey num = 0.005705711 Billion total entry num = 0.005705711 Billion write speed all = 0.023126105 MB/sec (with seq num) write speed all = 0.021862703 MB/sec (without seq num) 2024/11/22-18:54:32.330483 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221711: 923294 keys, 37851404 bytes, temperature: kWarm 2024/11/22-18:54:32.330551 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272872330517, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221711, "file_size": 37851404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3480995600, "table_properties": {"data_size": 31789776, "index_size": 1756736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25852232, "raw_average_key_size": 28, "raw_value_size": 108365520, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923294, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272869, "oldest_key_time": 0, "file_creation_time": 1732272869, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221711, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:32.366000 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221712.sst) /dev/shm/ToplingTemp/Topling-248533-9Rwe64 2024/11/22-18:54:33.147851 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221712.sst): first pass time = 0.78's, 171.676'MB/sec 2024/11/22-18:54:34.466193 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221712.sst): old prealloc_size = 147639500, real_size = 33566656 2024/11/22-18:54:34.466734 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221712.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922959 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.42 avg-zval = 34.20 2024/11/22-18:54:34.467492 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221712.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:35.400102 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221712.sst): second pass time = 1.32's, 82.245'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.197'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3420.146'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 298.174'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 25271.603'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.245'MB/sec zip pipeline throughput = 82.245'MB/sec entries = 922959 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.42 avg-zval = 34.20 usrkeys = 922959 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.42 avg-zval = 34.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922959 bytes = 3115008 } seq expand size = 6460713 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4434 value = 3.4330 dict = 1.00 all = 3.5754 } Zip/UnZip{ index = 0.0958 value = 0.2913 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 0.790603 GB avg = 0.119 KB (by entry num) total key len = 0.185603 GB avg = 0.028 KB total ukey len = 0.132573 GB avg = 0.020 KB total ukey num = 0.006628670 Billion total entry num = 0.006628670 Billion write speed all = 0.027017547 MB/sec (with seq num) write speed all = 0.025549903 MB/sec (without seq num) 2024/11/22-18:54:35.432802 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221712: 922959 keys, 37539748 bytes, temperature: kWarm 2024/11/22-18:54:35.432865 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272875432831, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221712, "file_size": 37539748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3480995283, "table_properties": {"data_size": 31568176, "index_size": 1767552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842852, "raw_average_key_size": 28, "raw_value_size": 108374946, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922959, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272872, "oldest_key_time": 0, "file_creation_time": 1732272872, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221712, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:35.470743 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221713.sst) /dev/shm/ToplingTemp/Topling-248533-rjM9g6 2024/11/22-18:54:36.281143 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221713.sst): first pass time = 0.81's, 165.629'MB/sec 2024/11/22-18:54:37.630450 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221713.sst): old prealloc_size = 147639500, real_size = 32984944 2024/11/22-18:54:37.630985 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221713.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922855 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0310 GB avg-val = 117.44 avg-zval = 33.59 2024/11/22-18:54:37.631737 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221713.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:38.536486 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221713.sst): second pass time = 1.35's, 80.361'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.919'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3482.356'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 295.745'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30037.569'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.361'MB/sec zip pipeline throughput = 80.361'MB/sec entries = 922855 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 33.59 usrkeys = 922855 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.44 avg-zval = 33.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922855 bytes = 3230000 } seq expand size = 6459985 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5122 value = 3.4963 dict = 1.00 all = 3.6204 } Zip/UnZip{ index = 0.0951 value = 0.2860 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 0.905441 GB avg = 0.120 KB (by entry num) total key len = 0.211443 GB avg = 0.028 KB total ukey len = 0.151031 GB avg = 0.020 KB total ukey num = 0.007551525 Billion total entry num = 0.007551525 Billion write speed all = 0.030908272 MB/sec (with seq num) write speed all = 0.029236445 MB/sec (without seq num) 2024/11/22-18:54:38.569848 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221713: 922855 keys, 37072586 bytes, temperature: kWarm 2024/11/22-18:54:38.569911 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272878569879, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221713, "file_size": 37072586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3480996338, "table_properties": {"data_size": 30998240, "index_size": 1755776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25839940, "raw_average_key_size": 28, "raw_value_size": 108377882, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922855, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272875, "oldest_key_time": 0, "file_creation_time": 1732272875, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221713, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:38.604486 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221714.sst) /dev/shm/ToplingTemp/Topling-248533-NF1Hd8 2024/11/22-18:54:39.562443 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221714.sst): first pass time = 0.96's, 140.113'MB/sec 2024/11/22-18:54:40.908643 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221714.sst): old prealloc_size = 147639500, real_size = 33807744 2024/11/22-18:54:40.909226 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221714.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923018 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.41 avg-zval = 34.46 2024/11/22-18:54:40.909979 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221714.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:41.837128 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221714.sst): second pass time = 1.35's, 80.572'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.816'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3166.143'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 298.604'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32046.100'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.572'MB/sec zip pipeline throughput = 80.572'MB/sec entries = 923018 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.41 avg-zval = 34.46 usrkeys = 923018 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.41 avg-zval = 34.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923018 bytes = 3230576 } seq expand size = 6461126 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4418 value = 3.4070 dict = 1.00 all = 3.5417 } Zip/UnZip{ index = 0.0958 value = 0.2935 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 1.020276 GB avg = 0.120 KB (by entry num) total key len = 0.237287 GB avg = 0.028 KB total ukey len = 0.169491 GB avg = 0.020 KB total ukey num = 0.008474543 Billion total entry num = 0.008474543 Billion write speed all = 0.034798193 MB/sec (with seq num) write speed all = 0.032922192 MB/sec (without seq num) 2024/11/22-18:54:41.870823 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221714: 923018 keys, 37896486 bytes, temperature: kWarm 2024/11/22-18:54:41.870892 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272881870855, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221714, "file_size": 37896486, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701122, "largest_seqno": 3480995316, "table_properties": {"data_size": 31808880, "index_size": 1767936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844504, "raw_average_key_size": 28, "raw_value_size": 108373248, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923018, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272878, "oldest_key_time": 0, "file_creation_time": 1732272878, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221714, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:41.907958 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221715.sst) /dev/shm/ToplingTemp/Topling-248533-Hwlw96 2024/11/22-18:54:42.727198 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221715.sst): first pass time = 0.82's, 163.841'MB/sec 2024/11/22-18:54:44.052431 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221715.sst): old prealloc_size = 147639500, real_size = 33375120 2024/11/22-18:54:44.052979 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221715.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923799 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0314 GB avg-val = 117.29 avg-zval = 33.98 2024/11/22-18:54:44.053710 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221715.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:44.983716 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221715.sst): second pass time = 1.32's, 81.830'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.107'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3347.742'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 292.298'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30864.302'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.830'MB/sec zip pipeline throughput = 81.830'MB/sec entries = 923799 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.29 avg-zval = 33.98 usrkeys = 923799 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.29 avg-zval = 33.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923799 bytes = 3117840 } seq expand size = 6466593 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5122 value = 3.4522 dict = 1.00 all = 3.5935 } Zip/UnZip{ index = 0.0951 value = 0.2897 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 1.135094 GB avg = 0.121 KB (by entry num) total key len = 0.263154 GB avg = 0.028 KB total ukey len = 0.187967 GB avg = 0.020 KB total ukey num = 0.009398342 Billion total entry num = 0.009398342 Billion write speed all = 0.038687722 MB/sec (with seq num) write speed all = 0.036607401 MB/sec (without seq num) 2024/11/22-18:54:45.016656 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221715: 923799 keys, 37350514 bytes, temperature: kWarm 2024/11/22-18:54:45.016721 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272885016686, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221715, "file_size": 37350514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3480995472, "table_properties": {"data_size": 31386432, "index_size": 1757568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25866372, "raw_average_key_size": 28, "raw_value_size": 108351446, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923799, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272881, "oldest_key_time": 0, "file_creation_time": 1732272881, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221715, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:45.056542 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221716.sst) /dev/shm/ToplingTemp/Topling-248533-pPdXt5 2024/11/22-18:54:45.846095 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221716.sst): first pass time = 0.79's, 170.001'MB/sec 2024/11/22-18:54:47.157810 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221716.sst): old prealloc_size = 147639500, real_size = 32205216 2024/11/22-18:54:47.158330 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221716.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922490 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0302 GB avg-val = 117.50 avg-zval = 32.76 2024/11/22-18:54:47.158937 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221716.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:48.038027 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221716.sst): second pass time = 1.31's, 82.675'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.352'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3430.778'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 272.116'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30832.323'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.675'MB/sec zip pipeline throughput = 82.675'MB/sec entries = 922490 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.50 avg-zval = 32.76 usrkeys = 922490 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.50 avg-zval = 32.76 TagRS{ kinds = RS_Key0_TagN bytes = 144240 } TagArray{ size = 659935 bytes = 2309792 } seq expand size = 4619545 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5115 value = 3.5867 dict = 1.00 all = 3.7789 } Zip/UnZip{ index = 0.0951 value = 0.2788 dict = 1.00 all = 0.2646 } ---------------------------- total value len = 1.248101 GB avg = 0.121 KB (by entry num) total key len = 0.288983 GB avg = 0.028 KB total ukey len = 0.206417 GB avg = 0.020 KB total ukey num = 0.010320832 Billion total entry num = 0.010320832 Billion write speed all = 0.042525581 MB/sec (with seq num) write speed all = 0.040241260 MB/sec (without seq num) 2024/11/22-18:54:48.069256 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221716: 922490 keys, 35517680 bytes, temperature: kWarm 2024/11/22-18:54:48.069318 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272888069284, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221716, "file_size": 35517680, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3480996548, "table_properties": {"data_size": 30219216, "index_size": 1755200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25829720, "raw_average_key_size": 28, "raw_value_size": 108388072, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922490, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272885, "oldest_key_time": 0, "file_creation_time": 1732272885, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221716, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:48.110807 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221717.sst) /dev/shm/ToplingTemp/Topling-248533-ddsrX4 2024/11/22-18:54:48.988008 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221717.sst): first pass time = 0.88's, 153.013'MB/sec 2024/11/22-18:54:50.333676 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221717.sst): old prealloc_size = 147639500, real_size = 33751552 2024/11/22-18:54:50.334239 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221717.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926045 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0318 GB avg-val = 116.94 avg-zval = 34.29 2024/11/22-18:54:50.335000 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221717.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:51.243341 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221717.sst): second pass time = 1.35's, 80.512'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.693'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3248.358'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 285.304'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36731.828'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.512'MB/sec zip pipeline throughput = 80.512'MB/sec entries = 926045 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 34.29 usrkeys = 926045 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 34.29 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 907059 bytes = 3174720 } seq expand size = 6349413 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5141 value = 3.4098 dict = 1.00 all = 3.5390 } Zip/UnZip{ index = 0.0951 value = 0.2933 dict = 1.00 all = 0.2826 } ---------------------------- total value len = 1.362739 GB avg = 0.121 KB (by entry num) total key len = 0.314913 GB avg = 0.028 KB total ukey len = 0.224938 GB avg = 0.020 KB total ukey num = 0.011246877 Billion total entry num = 0.011246877 Billion write speed all = 0.046410453 MB/sec (with seq num) write speed all = 0.043921390 MB/sec (without seq num) 2024/11/22-18:54:51.279264 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221717: 926045 keys, 37925822 bytes, temperature: kWarm 2024/11/22-18:54:51.279335 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272891279302, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221717, "file_size": 37925822, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3480996480, "table_properties": {"data_size": 31758320, "index_size": 1761536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25929260, "raw_average_key_size": 28, "raw_value_size": 108288570, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926045, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272888, "oldest_key_time": 0, "file_creation_time": 1732272888, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221717, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:51.318580 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221718.sst) /dev/shm/ToplingTemp/Topling-248533-kfwaN7 2024/11/22-18:54:52.148509 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221718.sst): first pass time = 0.83's, 161.730'MB/sec 2024/11/22-18:54:53.438516 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221718.sst): old prealloc_size = 147639500, real_size = 33681824 2024/11/22-18:54:53.439093 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221718.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922939 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.42 avg-zval = 34.34 2024/11/22-18:54:53.439856 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221718.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:54.374555 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221718.sst): second pass time = 1.29's, 84.054'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.229'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3167.909'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 293.465'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 29735.673'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.054'MB/sec zip pipeline throughput = 84.054'MB/sec entries = 922939 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 34.34 usrkeys = 922939 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.42 avg-zval = 34.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922939 bytes = 3114928 } seq expand size = 6460573 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5116 value = 3.4193 dict = 1.00 all = 3.5644 } Zip/UnZip{ index = 0.0951 value = 0.2925 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 1.477575 GB avg = 0.121 KB (by entry num) total key len = 0.340755 GB avg = 0.028 KB total ukey len = 0.243396 GB avg = 0.020 KB total ukey num = 0.012169816 Billion total entry num = 0.012169816 Billion write speed all = 0.050297813 MB/sec (with seq num) write speed all = 0.047604727 MB/sec (without seq num) 2024/11/22-18:54:54.408442 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221718: 922939 keys, 37654676 bytes, temperature: kWarm 2024/11/22-18:54:54.408507 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272894408474, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221718, "file_size": 37654676, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3480996026, "table_properties": {"data_size": 31694864, "index_size": 1756032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842292, "raw_average_key_size": 28, "raw_value_size": 108375494, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922939, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272891, "oldest_key_time": 0, "file_creation_time": 1732272891, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221718, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:54.443768 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221719.sst) /dev/shm/ToplingTemp/Topling-248533-7u4Db5 2024/11/22-18:54:55.311841 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221719.sst): first pass time = 0.87's, 154.623'MB/sec 2024/11/22-18:54:56.834038 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221719.sst): old prealloc_size = 147639500, real_size = 36305696 2024/11/22-18:54:56.834600 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221719.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928896 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0343 GB avg-val = 116.49 avg-zval = 36.91 2024/11/22-18:54:56.834914 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221719.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:54:57.544148 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221719.sst): second pass time = 1.52's, 71.140'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.695'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3320.778'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.399'MB/sec write SST data time = 0.13's, 298.235'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 24058.793'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.140'MB/sec zip pipeline throughput = 71.140'MB/sec entries = 928896 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.49 avg-zval = 36.91 usrkeys = 928896 avg-key = 20.00 avg-zkey = 1.93 avg-val = 118.26 avg-zval = 36.91 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 234597 bytes = 821104 } seq expand size = 1642179 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.3734 value = 3.1564 dict = 1.00 all = 3.5201 } Zip/UnZip{ index = 0.0964 value = 0.3168 dict = 1.00 all = 0.2841 } ---------------------------- total value len = 1.587426 GB avg = 0.121 KB (by entry num) total key len = 0.366764 GB avg = 0.028 KB total ukey len = 0.261974 GB avg = 0.020 KB total ukey num = 0.013098712 Billion total entry num = 0.013098712 Billion write speed all = 0.054051170 MB/sec (with seq num) write speed all = 0.051152780 MB/sec (without seq num) 2024/11/22-18:54:57.579515 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221719: 928896 keys, 38128917 bytes, temperature: kWarm 2024/11/22-18:54:57.579581 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272897579548, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221719, "file_size": 38128917, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3480995404, "table_properties": {"data_size": 34282384, "index_size": 1790912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009088, "raw_average_key_size": 28, "raw_value_size": 108208700, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928896, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272894, "oldest_key_time": 0, "file_creation_time": 1732272894, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221719, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:57.619765 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221720.sst) /dev/shm/ToplingTemp/Topling-248533-0zldT5 2024/11/22-18:54:57.887220 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221720.sst): first pass time = 0.27's, 246.576'MB/sec 2024/11/22-18:54:58.530199 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221720.sst): old prealloc_size = 147639500, real_size = 17195600 2024/11/22-18:54:58.530507 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221720.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 451626 prefix = 4 raw-key = 0.0072 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0533 GB zip-val = 0.0162 GB avg-val = 118.00 avg-zval = 35.91 2024/11/22-18:54:58.530925 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221720.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.066 GB 2024/11/22-18:54:59.004263 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00/221720.sst): second pass time = 0.64's, 82.973'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.47's, 40.508'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3018.192'MB/sec (index lex order gen) rebuild zvType time = 0.41's, 0.275'MB/sec write SST data time = 0.06's, 302.492'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.537'MB, throughput = 17166.928'MB/sec zip my value time = 0.64's, unzip length = 0.053'GB zip my value throughput = 82.973'MB/sec zip pipeline throughput = 82.973'MB/sec entries = 451626 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 35.91 usrkeys = 451626 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 35.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 451626 bytes = 1580704 } seq expand size = 3161382 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0090 GB value = 0.0533 GB dict = 0.54 MB all = 0.0659 GB } __ZipSize{ index = 0.0009 GB value = 0.0162 GB dict = 0.54 MB all = 0.0192 GB } UnZip/Zip{ index = 10.4396 value = 3.2861 dict = 1.00 all = 3.4337 } Zip/UnZip{ index = 0.0958 value = 0.3043 dict = 1.00 all = 0.2912 } ---------------------------- total value len = 1.643880 GB avg = 0.121 KB (by entry num) total key len = 0.379409 GB avg = 0.028 KB total ukey len = 0.271007 GB avg = 0.020 KB total ukey num = 0.013550338 Billion total entry num = 0.013550338 Billion write speed all = 0.055960121 MB/sec (with seq num) write speed all = 0.052961919 MB/sec (without seq num) 2024/11/22-18:54:59.024448 140025104320064 [compaction_job.cc:1948] [default] [JOB 8108] Generated table #221720: 451626 keys, 19203087 bytes, temperature: kWarm 2024/11/22-18:54:59.024511 140025104320064 EVENT_LOG_v1 {"time_micros": 1732272899024480, "cf_name": "default", "job": 8108, "event": "table_file_creation", "file_number": 221720, "file_size": 19203087, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273852071, "largest_seqno": 3480931875, "table_properties": {"data_size": 16217280, "index_size": 865216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12645528, "raw_average_key_size": 28, "raw_value_size": 53291868, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 451626, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272897, "oldest_key_time": 0, "file_creation_time": 1732272897, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221720, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:54:59.042693 140025104320064 [compaction_job.cc:703] job-08108: subcompact[0], size: 0.540978 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08108/att-00 [221706,221707,221708,221709,221710,221711,221712,221713,221714,221715,221716,221717,221718,221719,221720] 2024/11/22-18:54:59.134206 140025104320064 (Original Log Time 2024/11/22-18:54:13.296038) EVENT_LOG_v1 {"time_micros": 1732272853295998, "job": 8108, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [221643], "files_L3": [221700, 221701, 221650, 221599, 221600, 221601, 221602, 221603, 221604, 221605, 221606, 221573, 221574, 221575, 221576, 221577, 221578], "score": 0.971856, "input_data_size": 564153749, "oldest_snapshot_seqno": 3481759239} 2024/11/22-18:54:59.134209 140025104320064 (Original Log Time 2024/11/22-18:54:59.132760) [compaction_job.cc:2022] [default] [JOB 8108] Compacted 1@2 + 17@3 files to L3 => 540977779 bytes 2024/11/22-18:54:59.134210 140025104320064 (Original Log Time 2024/11/22-18:54:59.134119) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 12.3 rd, 11.8 wr, level 3, files in(1, 17) out(15 +0 blob) MB in(27.1, 510.9 +0.0 blob) out(515.9 +0.0 blob), read-write-amplify(38.9) write-amplify(19.0) OK, records in: 13654367, records dropped: 104029 output_compression: Snappy 2024/11/22-18:54:59.134215 140025104320064 (Original Log Time 2024/11/22-18:54:59.134185) EVENT_LOG_v1 {"time_micros": 1732272899134143, "cf": "default", "job": 8108, "event": "compaction_finished", "compaction_time_micros": 45746665, "compaction_time_cpu_micros": 20688525, "output_level": 3, "num_output_files": 15, "total_output_size": 540977779, "num_input_records": 13654367, "num_output_records": 13550338, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 343647157, "file_range_sync_nanos": 0, "file_fsync_nanos": 16059, "file_prepare_write_nanos": 36444, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/22-18:54:59.271636 140025104320064 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732272853,"executesMs":45968,"compactionJobId":8108,"attempt":0,"compactionInputRawBytes":1959758451,"compactionInputZipBytes":564153749,"compactionOutputRawBytes":0,"compactionOutputZipBytes":540977779,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-0812b38ff0b3d37e853136015c59ed7a-a787f64d4a679132-00"}}, response = 2024/11/22-18:54:59.271701 140025104320064 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08108/att-00: olev 3, work 45.969 s, result 6.310 ms, install 1.482 ms, input{raw 1.825 GiB zip 538.019 MiB}