2024/11/24-04:46:09.034746 139957651584576 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00: fork to child time = 0.007257 sec 2024/11/24-04:46:09.035524 139957651584576 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00 2024/11/24-04:46:09.042086 139957651584576 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00 2024/11/24-04:46:09.042947 139957651584576 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-04:46:09.043216 139957651584576 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-04:46:09.105711 139957651584576 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41734/att-00/MANIFEST-206347 2024/11/24-04:46:09.121828 139957651584576 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41734/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 292437, last_sequence is 4445578715, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 292292 2024/11/24-04:46:09.121835 139957651584576 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-04:46:09.121961 139957651584576 [version_set.cc:5721] Creating manifest 292437 2024/11/24-04:46:09.157480 139957651584576 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-04:46:09.181989 139957651584576 [compaction_job.cc:2437] [default] [JOB 41734] Compacting 1@2 + 22@3 files to L3, score 0.66, subcompactions 1 : 1 2024/11/24-04:46:09.182011 139957651584576 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31281 Base level 2, inputs: [292428(27MB)], [292401(34MB) 292402(35MB) 292403(33MB) 292404(3821KB) 292363(33MB) 292364(35MB) 292365(34MB) 292366(32MB) 292367(30MB) 292368(35MB) 292369(32MB) 292370(33MB) 292371(33MB) 292372(33MB) 292373(33MB) 292374(32MB) 292375(33MB) 292376(32MB) 292377(34MB) 292378(33MB) 292379(35MB) 292380(33MB)] 2024/11/24-04:46:09.188808 139957651584576 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:46:09.188839 139957651584576 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:46:09.188859 139957651584576 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:46:09.188865 139957651584576 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:46:09.188877 139957651584576 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:46:09.767856 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292438.sst) /dev/shm/ToplingTemp/Topling-272834-TJx1FD 2024/11/24-04:46:10.339448 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292438.sst): first pass time = 0.57's, 234.829'MB/sec 2024/11/24-04:46:14.555565 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292438.sst): old prealloc_size = 147639500, real_size = 31690080 2024/11/24-04:46:14.556495 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292438.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919762 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0297 GB avg-val = 117.93 avg-zval = 32.30 2024/11/24-04:46:14.557480 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292438.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:15.454464 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292438.sst): second pass time = 4.21's, 25.734'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 39.793'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1976.258'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.303'MB/sec write SST data time = 0.14's, 256.290'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36209.687'MB/sec zip my value time = 4.21's, unzip length = 0.108'GB zip my value throughput = 25.734'MB/sec zip pipeline throughput = 90.125'MB/sec entries = 919762 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.93 avg-zval = 32.30 usrkeys = 919762 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.93 avg-zval = 32.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919762 bytes = 3219184 } seq expand size = 6438334 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5157 value = 3.6507 dict = 1.00 all = 3.7523 } Zip/UnZip{ index = 0.0951 value = 0.2739 dict = 1.00 all = 0.2665 } ---------------------------- total value len = 2251.440476 GB avg = 0.200 KB (by entry num) total key len = 306.908716 GB avg = 0.027 KB total ukey len = 237.373123 GB avg = 0.015 KB total ukey num = 15.739580925 Billion total entry num = 11.280894873 Billion write speed all = 35.799513127 MB/sec (with seq num) write speed all = 34.536665834 MB/sec (without seq num) 2024/11/24-04:46:15.501684 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292438: 919762 keys, 35769878 bytes, temperature: kWarm 2024/11/24-04:46:15.501775 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394775501718, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292438, "file_size": 35769878, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4445557751, "table_properties": {"data_size": 29710672, "index_size": 1749312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25753336, "raw_average_key_size": 28, "raw_value_size": 108464512, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919762, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394769, "oldest_key_time": 0, "file_creation_time": 1732394769, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292438, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:15.542269 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292439.sst) /dev/shm/ToplingTemp/Topling-272834-A6pbrC 2024/11/24-04:46:16.824021 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292439.sst): first pass time = 1.28's, 104.718'MB/sec 2024/11/24-04:46:18.405332 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292439.sst): old prealloc_size = 147639500, real_size = 33404944 2024/11/24-04:46:18.406043 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292439.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927163 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.76 avg-zval = 33.88 2024/11/24-04:46:18.406887 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292439.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:19.298202 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292439.sst): second pass time = 1.58's, 68.504'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 42.006'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2570.820'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.15's, 246.860'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34937.902'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.504'MB/sec zip pipeline throughput = 68.504'MB/sec entries = 927163 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 33.88 usrkeys = 927163 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 33.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927163 bytes = 3245088 } seq expand size = 6490141 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5126 value = 3.4467 dict = 1.00 all = 3.5787 } Zip/UnZip{ index = 0.0951 value = 0.2901 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 2251.555223 GB avg = 0.200 KB (by entry num) total key len = 306.934676 GB avg = 0.027 KB total ukey len = 237.391666 GB avg = 0.015 KB total ukey num = 15.740508088 Billion total entry num = 11.281822036 Billion write speed all = 35.799556562 MB/sec (with seq num) write speed all = 34.536673402 MB/sec (without seq num) 2024/11/24-04:46:19.345604 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292439: 927163 keys, 37504884 bytes, temperature: kWarm 2024/11/24-04:46:19.345661 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394779345630, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292439, "file_size": 37504884, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4445562917, "table_properties": {"data_size": 31409088, "index_size": 1763904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960564, "raw_average_key_size": 28, "raw_value_size": 108257202, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927163, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394775, "oldest_key_time": 0, "file_creation_time": 1732394775, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292439, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:19.382881 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292440.sst) /dev/shm/ToplingTemp/Topling-272834-rFWP2D 2024/11/24-04:46:20.708296 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292440.sst): first pass time = 1.33's, 101.268'MB/sec 2024/11/24-04:46:22.058068 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292440.sst): old prealloc_size = 147639500, real_size = 30640048 2024/11/24-04:46:22.058631 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292440.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926796 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0286 GB avg-val = 116.82 avg-zval = 30.90 2024/11/24-04:46:22.059441 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292440.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:22.896100 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292440.sst): second pass time = 1.35's, 80.287'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.314'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3261.015'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 270.231'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34724.538'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.287'MB/sec zip pipeline throughput = 80.287'MB/sec entries = 926796 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.82 avg-zval = 30.90 usrkeys = 926796 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.82 avg-zval = 30.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926796 bytes = 3127952 } seq expand size = 6487572 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4463 value = 3.7811 dict = 1.00 all = 3.8768 } Zip/UnZip{ index = 0.0957 value = 0.2645 dict = 1.00 all = 0.2579 } ---------------------------- total value len = 2251.669979 GB avg = 0.200 KB (by entry num) total key len = 306.960627 GB avg = 0.027 KB total ukey len = 237.410202 GB avg = 0.015 KB total ukey num = 15.741434884 Billion total entry num = 11.282748832 Billion write speed all = 35.799723098 MB/sec (with seq num) write speed all = 34.536799772 MB/sec (without seq num) 2024/11/24-04:46:22.939660 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292440: 926796 keys, 34620758 bytes, temperature: kWarm 2024/11/24-04:46:22.939727 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394782939694, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292440, "file_size": 34620758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4338272005, "largest_seqno": 4445562933, "table_properties": {"data_size": 28633760, "index_size": 1774400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950288, "raw_average_key_size": 28, "raw_value_size": 108267552, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926796, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394779, "oldest_key_time": 0, "file_creation_time": 1732394779, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292440, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:22.979539 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292441.sst) /dev/shm/ToplingTemp/Topling-272834-ICbQaD 2024/11/24-04:46:24.178527 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292441.sst): first pass time = 1.20's, 111.946'MB/sec 2024/11/24-04:46:25.530744 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292441.sst): old prealloc_size = 147639500, real_size = 31517584 2024/11/24-04:46:25.531289 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292441.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919729 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.93 avg-zval = 32.12 2024/11/24-04:46:25.532054 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292441.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:26.342355 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292441.sst): second pass time = 1.35's, 80.253'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.862'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3328.186'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 278.964'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33233.649'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.253'MB/sec zip pipeline throughput = 80.253'MB/sec entries = 919729 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.93 avg-zval = 32.12 usrkeys = 919729 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.93 avg-zval = 32.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919729 bytes = 3219072 } seq expand size = 6438103 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5122 value = 3.6721 dict = 1.00 all = 3.7704 } Zip/UnZip{ index = 0.0951 value = 0.2723 dict = 1.00 all = 0.2652 } ---------------------------- total value len = 2251.784882 GB avg = 0.200 KB (by entry num) total key len = 306.986379 GB avg = 0.027 KB total ukey len = 237.428596 GB avg = 0.015 KB total ukey num = 15.742354613 Billion total entry num = 11.283668561 Billion write speed all = 35.799964874 MB/sec (with seq num) write speed all = 34.536999499 MB/sec (without seq num) 2024/11/24-04:46:26.386157 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292441: 919729 keys, 35597374 bytes, temperature: kWarm 2024/11/24-04:46:26.386233 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394786386190, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292441, "file_size": 35597374, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4445563230, "table_properties": {"data_size": 29537664, "index_size": 1749824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25752412, "raw_average_key_size": 28, "raw_value_size": 108465458, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919729, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394782, "oldest_key_time": 0, "file_creation_time": 1732394782, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292441, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:26.424137 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292442.sst) /dev/shm/ToplingTemp/Topling-272834-oqW7rE 2024/11/24-04:46:27.494530 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292442.sst): first pass time = 1.07's, 125.396'MB/sec 2024/11/24-04:46:28.861590 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292442.sst): old prealloc_size = 147639500, real_size = 33477328 2024/11/24-04:46:28.862153 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292442.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926907 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.80 avg-zval = 33.97 2024/11/24-04:46:28.862916 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292442.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:29.724465 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292442.sst): second pass time = 1.37's, 79.265'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.549'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3256.722'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 296.922'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35411.059'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.265'MB/sec zip pipeline throughput = 79.265'MB/sec entries = 926907 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 33.97 usrkeys = 926907 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 33.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926907 bytes = 3244192 } seq expand size = 6488349 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5158 value = 3.4389 dict = 1.00 all = 3.5719 } Zip/UnZip{ index = 0.0951 value = 0.2908 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 2251.899635 GB avg = 0.200 KB (by entry num) total key len = 307.012332 GB avg = 0.027 KB total ukey len = 237.447134 GB avg = 0.015 KB total ukey num = 15.743281520 Billion total entry num = 11.284595468 Billion write speed all = 35.800239463 MB/sec (with seq num) write speed all = 34.537230106 MB/sec (without seq num) 2024/11/24-04:46:29.770958 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292442: 926907 keys, 37575988 bytes, temperature: kWarm 2024/11/24-04:46:29.771036 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394789770999, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292442, "file_size": 37575988, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4445562947, "table_properties": {"data_size": 31482560, "index_size": 1762880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953396, "raw_average_key_size": 28, "raw_value_size": 108264434, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926907, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394786, "oldest_key_time": 0, "file_creation_time": 1732394786, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292442, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:29.806326 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292443.sst) /dev/shm/ToplingTemp/Topling-272834-EwHlYD 2024/11/24-04:46:30.985514 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292443.sst): first pass time = 1.18's, 113.825'MB/sec 2024/11/24-04:46:32.321746 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292443.sst): old prealloc_size = 147639500, real_size = 30939648 2024/11/24-04:46:32.322350 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292443.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927324 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 116.74 avg-zval = 31.20 2024/11/24-04:46:32.323148 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292443.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:33.139399 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292443.sst): second pass time = 1.34's, 81.060'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.860'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3056.236'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.12's, 284.475'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28864.707'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.060'MB/sec zip pipeline throughput = 81.060'MB/sec entries = 927324 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.74 avg-zval = 31.20 usrkeys = 927324 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.74 avg-zval = 31.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927324 bytes = 3245648 } seq expand size = 6491268 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4432 value = 3.7417 dict = 1.00 all = 3.8301 } Zip/UnZip{ index = 0.0958 value = 0.2673 dict = 1.00 all = 0.2611 } ---------------------------- total value len = 2252.014379 GB avg = 0.200 KB (by entry num) total key len = 307.038298 GB avg = 0.027 KB total ukey len = 237.465681 GB avg = 0.015 KB total ukey num = 15.744208844 Billion total entry num = 11.285522792 Billion write speed all = 35.800497624 MB/sec (with seq num) write speed all = 34.537444821 MB/sec (without seq num) 2024/11/24-04:46:33.183653 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292443: 927324 keys, 35043302 bytes, temperature: kWarm 2024/11/24-04:46:33.183739 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394793183686, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292443, "file_size": 35043302, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4445562964, "table_properties": {"data_size": 28931696, "index_size": 1775936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965072, "raw_average_key_size": 28, "raw_value_size": 108252744, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927324, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394789, "oldest_key_time": 0, "file_creation_time": 1732394789, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292443, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:33.218772 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292444.sst) /dev/shm/ToplingTemp/Topling-272834-ISguVE 2024/11/24-04:46:34.146448 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292444.sst): first pass time = 0.93's, 144.687'MB/sec 2024/11/24-04:46:35.503655 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292444.sst): old prealloc_size = 147639500, real_size = 30718560 2024/11/24-04:46:35.504222 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292444.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926901 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0287 GB avg-val = 116.80 avg-zval = 30.96 2024/11/24-04:46:35.505017 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292444.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:36.319347 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292444.sst): second pass time = 1.36's, 79.811'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.683'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3201.474'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 277.993'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36025.342'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.811'MB/sec zip pipeline throughput = 79.811'MB/sec entries = 926901 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.80 avg-zval = 30.96 usrkeys = 926901 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.80 avg-zval = 30.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926901 bytes = 3244176 } seq expand size = 6488307 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.3771 value = 3.7723 dict = 1.00 all = 3.8551 } Zip/UnZip{ index = 0.0964 value = 0.2651 dict = 1.00 all = 0.2594 } ---------------------------- total value len = 2252.129132 GB avg = 0.200 KB (by entry num) total key len = 307.064251 GB avg = 0.027 KB total ukey len = 237.484219 GB avg = 0.015 KB total ukey num = 15.745135745 Billion total entry num = 11.286449693 Billion write speed all = 35.800873404 MB/sec (with seq num) write speed all = 34.537773055 MB/sec (without seq num) 2024/11/24-04:46:36.362331 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292444: 926901 keys, 34815392 bytes, temperature: kWarm 2024/11/24-04:46:36.362398 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394796362364, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292444, "file_size": 34815392, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4445558880, "table_properties": {"data_size": 28700240, "index_size": 1786432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953228, "raw_average_key_size": 28, "raw_value_size": 108264534, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926901, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394793, "oldest_key_time": 0, "file_creation_time": 1732394793, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292444, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:36.398991 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292445.sst) /dev/shm/ToplingTemp/Topling-272834-KY4D3A 2024/11/24-04:46:37.345866 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292445.sst): first pass time = 0.95's, 141.754'MB/sec 2024/11/24-04:46:38.705135 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292445.sst): old prealloc_size = 147639500, real_size = 27721072 2024/11/24-04:46:38.705673 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292445.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.14 2024/11/24-04:46:38.706434 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292445.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:39.556223 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292445.sst): second pass time = 1.36's, 79.316'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 37.309'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3547.170'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.326'MB/sec write SST data time = 0.13's, 250.731'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 36774.949'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.316'MB/sec zip pipeline throughput = 79.316'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.94 avg-val = 114.00 avg-zval = 27.14 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.94 avg-val = 121.00 avg-zval = 27.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.3108 value = 4.2007 dict = 1.00 all = 4.2270 } Zip/UnZip{ index = 0.0970 value = 0.2381 dict = 1.00 all = 0.2366 } ---------------------------- total value len = 2252.243500 GB avg = 0.200 KB (by entry num) total key len = 307.090716 GB avg = 0.027 KB total ukey len = 237.503123 GB avg = 0.015 KB total ukey num = 15.746080941 Billion total entry num = 11.287394889 Billion write speed all = 35.801222433 MB/sec (with seq num) write speed all = 34.538073501 MB/sec (without seq num) 2024/11/24-04:46:39.596438 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292445: 945196 keys, 31752368 bytes, temperature: kWarm 2024/11/24-04:46:39.596504 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394799596472, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292445, "file_size": 31752368, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4294448541, "largest_seqno": 4395780648, "table_properties": {"data_size": 25651168, "index_size": 1833408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394796, "oldest_key_time": 0, "file_creation_time": 1732394796, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292445, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:39.632509 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292446.sst) /dev/shm/ToplingTemp/Topling-272834-IwAIeB 2024/11/24-04:46:40.656357 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292446.sst): first pass time = 1.02's, 131.097'MB/sec 2024/11/24-04:46:42.148560 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292446.sst): old prealloc_size = 147639500, real_size = 32871488 2024/11/24-04:46:42.149134 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292446.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931779 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.04 avg-zval = 33.13 2024/11/24-04:46:42.149908 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292446.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:43.018218 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292446.sst): second pass time = 1.49's, 72.509'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.528'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3220.006'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 290.821'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33968.167'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.509'MB/sec zip pipeline throughput = 72.509'MB/sec entries = 931779 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.04 avg-zval = 33.13 usrkeys = 931779 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.04 avg-zval = 33.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931779 bytes = 3261248 } seq expand size = 6522453 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5119 value = 3.5032 dict = 1.00 all = 3.6291 } Zip/UnZip{ index = 0.0951 value = 0.2855 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 2252.358151 GB avg = 0.200 KB (by entry num) total key len = 307.116806 GB avg = 0.027 KB total ukey len = 237.521758 GB avg = 0.015 KB total ukey num = 15.747012720 Billion total entry num = 11.288326668 Billion write speed all = 35.801457381 MB/sec (with seq num) write speed all = 34.538265350 MB/sec (without seq num) 2024/11/24-04:46:43.065614 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292446: 931779 keys, 36983422 bytes, temperature: kWarm 2024/11/24-04:46:43.065683 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394803065649, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292446, "file_size": 36983422, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4445562589, "table_properties": {"data_size": 30865584, "index_size": 1772800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26089812, "raw_average_key_size": 28, "raw_value_size": 108127974, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931779, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394799, "oldest_key_time": 0, "file_creation_time": 1732394799, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292446, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:43.102475 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292447.sst) /dev/shm/ToplingTemp/Topling-272834-2Vw0ED 2024/11/24-04:46:44.082880 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292447.sst): first pass time = 0.98's, 136.906'MB/sec 2024/11/24-04:46:45.557777 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292447.sst): old prealloc_size = 147639500, real_size = 30823632 2024/11/24-04:46:45.558361 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292447.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926407 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.88 avg-zval = 31.11 2024/11/24-04:46:45.559113 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292447.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:46.386884 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292447.sst): second pass time = 1.47's, 73.479'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.978'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3196.224'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.12's, 280.741'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32620.604'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.479'MB/sec zip pipeline throughput = 73.479'MB/sec entries = 926407 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.88 avg-zval = 31.11 usrkeys = 926407 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.88 avg-zval = 31.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926407 bytes = 3126640 } seq expand size = 6484849 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4442 value = 3.7573 dict = 1.00 all = 3.8565 } Zip/UnZip{ index = 0.0957 value = 0.2661 dict = 1.00 all = 0.2593 } ---------------------------- total value len = 2252.472914 GB avg = 0.200 KB (by entry num) total key len = 307.142745 GB avg = 0.027 KB total ukey len = 237.540287 GB avg = 0.015 KB total ukey num = 15.747939127 Billion total entry num = 11.289253075 Billion write speed all = 35.801738517 MB/sec (with seq num) write speed all = 34.538502343 MB/sec (without seq num) 2024/11/24-04:46:46.430767 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292447: 926407 keys, 34803328 bytes, temperature: kWarm 2024/11/24-04:46:46.430836 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394806430801, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292447, "file_size": 34803328, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4328080174, "largest_seqno": 4445560138, "table_properties": {"data_size": 28817856, "index_size": 1774016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939396, "raw_average_key_size": 28, "raw_value_size": 108278410, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926407, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394803, "oldest_key_time": 0, "file_creation_time": 1732394803, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292447, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:46.483342 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292448.sst) /dev/shm/ToplingTemp/Topling-272834-21nNMB 2024/11/24-04:46:47.485894 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292448.sst): first pass time = 1.00's, 133.881'MB/sec 2024/11/24-04:46:48.932163 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292448.sst): old prealloc_size = 147639500, real_size = 29863584 2024/11/24-04:46:48.932716 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292448.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 941845 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0278 GB avg-val = 114.51 avg-zval = 29.54 2024/11/24-04:46:48.933486 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292448.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:49.771042 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292448.sst): second pass time = 1.45's, 74.616'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 40.540'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3407.116'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.329'MB/sec write SST data time = 0.12's, 276.077'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.076'MB, throughput = 35745.273'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.616'MB/sec zip pipeline throughput = 74.616'MB/sec entries = 941845 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.51 avg-zval = 29.54 usrkeys = 941845 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.51 avg-zval = 29.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 941845 bytes = 3296480 } seq expand size = 6592915 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1078 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0278 GB dict = 1.08 MB all = 0.0340 GB } UnZip/Zip{ index = 10.4441 value = 3.8760 dict = 1.00 all = 3.9469 } Zip/UnZip{ index = 0.0957 value = 0.2580 dict = 1.00 all = 0.2534 } ---------------------------- total value len = 2252.587353 GB avg = 0.200 KB (by entry num) total key len = 307.169117 GB avg = 0.027 KB total ukey len = 237.559124 GB avg = 0.015 KB total ukey num = 15.748880972 Billion total entry num = 11.290194920 Billion write speed all = 35.802013387 MB/sec (with seq num) write speed all = 34.538731619 MB/sec (without seq num) 2024/11/24-04:46:49.813486 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292448: 941845 keys, 34006202 bytes, temperature: kWarm 2024/11/24-04:46:49.813551 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394809813519, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292448, "file_size": 34006202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4445562735, "table_properties": {"data_size": 27824336, "index_size": 1803584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26371660, "raw_average_key_size": 28, "raw_value_size": 107846130, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 941845, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394806, "oldest_key_time": 0, "file_creation_time": 1732394806, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292448, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:49.849769 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292449.sst) /dev/shm/ToplingTemp/Topling-272834-MCsuKB 2024/11/24-04:46:50.777575 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292449.sst): first pass time = 0.93's, 144.667'MB/sec 2024/11/24-04:46:52.241134 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292449.sst): old prealloc_size = 147639500, real_size = 31344640 2024/11/24-04:46:52.241665 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292449.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.93 2024/11/24-04:46:52.242410 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292449.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:53.049800 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292449.sst): second pass time = 1.46's, 74.176'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.662'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3468.858'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.12's, 287.613'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34045.938'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.176'MB/sec zip pipeline throughput = 74.176'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.93 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4462 value = 3.6954 dict = 1.00 all = 3.8015 } Zip/UnZip{ index = 0.0957 value = 0.2706 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 2252.702266 GB avg = 0.200 KB (by entry num) total key len = 307.194858 GB avg = 0.027 KB total ukey len = 237.577510 GB avg = 0.015 KB total ukey num = 15.749800272 Billion total entry num = 11.291114220 Billion write speed all = 35.802338785 MB/sec (with seq num) write speed all = 34.539012089 MB/sec (without seq num) 2024/11/24-04:46:53.093470 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292449: 919300 keys, 35306482 bytes, temperature: kWarm 2024/11/24-04:46:53.093542 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394813093507, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292449, "file_size": 35306482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4279740299, "largest_seqno": 4396863410, "table_properties": {"data_size": 29354544, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394809, "oldest_key_time": 0, "file_creation_time": 1732394809, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292449, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:53.129527 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292450.sst) /dev/shm/ToplingTemp/Topling-272834-6Fe64B 2024/11/24-04:46:53.993780 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292450.sst): first pass time = 0.86's, 155.306'MB/sec 2024/11/24-04:46:55.432891 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292450.sst): old prealloc_size = 147639500, real_size = 29588864 2024/11/24-04:46:55.433454 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292450.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934741 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0276 GB avg-val = 115.59 avg-zval = 29.49 2024/11/24-04:46:55.434173 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292450.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:56.252722 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292450.sst): second pass time = 1.44's, 75.174'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 40.833'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3297.678'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.12's, 270.125'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35964.578'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.174'MB/sec zip pipeline throughput = 75.174'MB/sec entries = 934741 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.59 avg-zval = 29.49 usrkeys = 934741 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.59 avg-zval = 29.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934741 bytes = 3037920 } seq expand size = 6543187 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0276 GB dict = 1.08 MB all = 0.0335 GB } UnZip/Zip{ index = 10.4421 value = 3.9197 dict = 1.00 all = 4.0095 } Zip/UnZip{ index = 0.0958 value = 0.2551 dict = 1.00 all = 0.2494 } ---------------------------- total value len = 2252.816854 GB avg = 0.200 KB (by entry num) total key len = 307.221030 GB avg = 0.027 KB total ukey len = 237.596204 GB avg = 0.015 KB total ukey num = 15.750735013 Billion total entry num = 11.292048961 Billion write speed all = 35.802703643 MB/sec (with seq num) write speed all = 34.539328956 MB/sec (without seq num) 2024/11/24-04:46:56.294467 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292450: 934741 keys, 33475326 bytes, temperature: kWarm 2024/11/24-04:46:56.294532 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394816294498, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292450, "file_size": 33475326, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4379543501, "largest_seqno": 4445560151, "table_properties": {"data_size": 27564656, "index_size": 1790336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26172748, "raw_average_key_size": 28, "raw_value_size": 108045110, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934741, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394813, "oldest_key_time": 0, "file_creation_time": 1732394813, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292450, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:56.331448 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292451.sst) /dev/shm/ToplingTemp/Topling-272834-zKF3KE 2024/11/24-04:46:57.163872 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292451.sst): first pass time = 0.83's, 161.245'MB/sec 2024/11/24-04:46:58.556341 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292451.sst): old prealloc_size = 147639500, real_size = 31883744 2024/11/24-04:46:58.556891 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292451.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920557 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0299 GB avg-val = 117.80 avg-zval = 32.48 2024/11/24-04:46:58.557640 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292451.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:46:59.375131 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292451.sst): second pass time = 1.39's, 77.942'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.927'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3334.544'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.12's, 289.711'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35654.676'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.942'MB/sec zip pipeline throughput = 77.942'MB/sec entries = 920557 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.80 avg-zval = 32.48 usrkeys = 920557 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.80 avg-zval = 32.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920557 bytes = 3221968 } seq expand size = 6443899 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5129 value = 3.6266 dict = 1.00 all = 3.7318 } Zip/UnZip{ index = 0.0951 value = 0.2757 dict = 1.00 all = 0.2680 } ---------------------------- total value len = 2252.931740 GB avg = 0.199 KB (by entry num) total key len = 307.246806 GB avg = 0.027 KB total ukey len = 237.614615 GB avg = 0.015 KB total ukey num = 15.751655570 Billion total entry num = 11.292969518 Billion write speed all = 35.803107392 MB/sec (with seq num) write speed all = 34.539684882 MB/sec (without seq num) 2024/11/24-04:46:59.420204 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292451: 920557 keys, 35965690 bytes, temperature: kWarm 2024/11/24-04:46:59.420278 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394819420244, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292451, "file_size": 35965690, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4445561573, "table_properties": {"data_size": 29902096, "index_size": 1751296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25775596, "raw_average_key_size": 28, "raw_value_size": 108442174, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920557, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394816, "oldest_key_time": 0, "file_creation_time": 1732394816, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292451, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:46:59.454412 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292452.sst) /dev/shm/ToplingTemp/Topling-272834-cROdbC 2024/11/24-04:47:00.319153 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292452.sst): first pass time = 0.86's, 155.219'MB/sec 2024/11/24-04:47:01.894170 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292452.sst): old prealloc_size = 147639500, real_size = 30493760 2024/11/24-04:47:01.894691 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292452.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927696 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0285 GB avg-val = 116.68 avg-zval = 30.71 2024/11/24-04:47:01.895447 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292452.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:47:02.713456 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292452.sst): second pass time = 1.57's, 68.756'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.083'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3542.613'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 267.792'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28285.288'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.756'MB/sec zip pipeline throughput = 121.329'MB/sec entries = 927696 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.68 avg-zval = 30.71 usrkeys = 927696 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.68 avg-zval = 30.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927696 bytes = 3130992 } seq expand size = 6493872 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4463 value = 3.7999 dict = 1.00 all = 3.8930 } Zip/UnZip{ index = 0.0957 value = 0.2632 dict = 1.00 all = 0.2569 } ---------------------------- total value len = 2253.046476 GB avg = 0.199 KB (by entry num) total key len = 307.272781 GB avg = 0.027 KB total ukey len = 237.633169 GB avg = 0.015 KB total ukey num = 15.752583266 Billion total entry num = 11.293897214 Billion write speed all = 35.803403700 MB/sec (with seq num) write speed all = 34.539936388 MB/sec (without seq num) 2024/11/24-04:47:02.794239 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292452: 927696 keys, 34477116 bytes, temperature: kWarm 2024/11/24-04:47:02.794286 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394822794259, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292452, "file_size": 34477116, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4369226147, "largest_seqno": 4445560164, "table_properties": {"data_size": 28485552, "index_size": 1776128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975488, "raw_average_key_size": 28, "raw_value_size": 108242328, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927696, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394819, "oldest_key_time": 0, "file_creation_time": 1732394819, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292452, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:47:02.844797 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292453.sst) /dev/shm/ToplingTemp/Topling-272834-M3iOvC 2024/11/24-04:47:03.677913 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292453.sst): first pass time = 0.83's, 161.114'MB/sec 2024/11/24-04:47:05.306624 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292453.sst): old prealloc_size = 147639500, real_size = 32705968 2024/11/24-04:47:05.307174 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292453.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923984 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.26 avg-zval = 33.23 2024/11/24-04:47:05.308523 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292453.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:47:06.140561 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292453.sst): second pass time = 1.63's, 66.557'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.126'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3362.979'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 290.882'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32622.459'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.557'MB/sec zip pipeline throughput = 128.502'MB/sec entries = 923984 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.26 avg-zval = 33.23 usrkeys = 923984 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.26 avg-zval = 33.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923984 bytes = 3233952 } seq expand size = 6467888 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4440 value = 3.5286 dict = 1.00 all = 3.6476 } Zip/UnZip{ index = 0.0957 value = 0.2834 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 2253.161290 GB avg = 0.199 KB (by entry num) total key len = 307.298653 GB avg = 0.027 KB total ukey len = 237.651649 GB avg = 0.015 KB total ukey num = 15.753507250 Billion total entry num = 11.294821198 Billion write speed all = 35.803655175 MB/sec (with seq num) write speed all = 34.540145049 MB/sec (without seq num) 2024/11/24-04:47:06.187230 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292453: 923984 keys, 36796594 bytes, temperature: kWarm 2024/11/24-04:47:06.187306 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394826187269, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292453, "file_size": 36796594, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4445562670, "table_properties": {"data_size": 30705376, "index_size": 1769408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25871552, "raw_average_key_size": 28, "raw_value_size": 108346240, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923984, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394822, "oldest_key_time": 0, "file_creation_time": 1732394822, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292453, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:47:06.230184 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292454.sst) /dev/shm/ToplingTemp/Topling-272834-Gh21VE 2024/11/24-04:47:07.245925 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292454.sst): first pass time = 1.02's, 132.146'MB/sec 2024/11/24-04:47:08.560371 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292454.sst): old prealloc_size = 147639500, real_size = 31014128 2024/11/24-04:47:08.560960 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292454.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923891 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 117.27 avg-zval = 31.39 2024/11/24-04:47:08.561762 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292454.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:47:09.376500 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292454.sst): second pass time = 1.31's, 82.482'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.871'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3026.870'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 274.152'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32211.237'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.482'MB/sec zip pipeline throughput = 82.482'MB/sec entries = 923891 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.27 avg-zval = 31.39 usrkeys = 923891 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.27 avg-zval = 31.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923891 bytes = 3118144 } seq expand size = 6467237 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.3776 value = 3.7359 dict = 1.00 all = 3.8362 } Zip/UnZip{ index = 0.0964 value = 0.2677 dict = 1.00 all = 0.2607 } ---------------------------- total value len = 2253.276106 GB avg = 0.199 KB (by entry num) total key len = 307.324522 GB avg = 0.027 KB total ukey len = 237.670127 GB avg = 0.015 KB total ukey num = 15.754431141 Billion total entry num = 11.295745089 Billion write speed all = 35.804002316 MB/sec (with seq num) write speed all = 34.540446011 MB/sec (without seq num) 2024/11/24-04:47:09.419637 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292454: 923891 keys, 34987530 bytes, temperature: kWarm 2024/11/24-04:47:09.419709 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394829419673, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292454, "file_size": 34987530, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4353496808, "largest_seqno": 4445560178, "table_properties": {"data_size": 29002400, "index_size": 1780544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868948, "raw_average_key_size": 28, "raw_value_size": 108348818, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923891, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394826, "oldest_key_time": 0, "file_creation_time": 1732394826, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292454, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:47:09.457152 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292455.sst) /dev/shm/ToplingTemp/Topling-272834-bVEzyD 2024/11/24-04:47:10.428860 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292455.sst): first pass time = 0.97's, 138.132'MB/sec 2024/11/24-04:47:11.746669 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292455.sst): old prealloc_size = 147639500, real_size = 33081456 2024/11/24-04:47:11.747293 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292455.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926747 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.83 avg-zval = 33.54 2024/11/24-04:47:11.748080 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292455.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:47:12.597803 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292455.sst): second pass time = 1.32's, 82.208'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.683'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2916.043'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 293.487'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34057.390'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.208'MB/sec zip pipeline throughput = 82.208'MB/sec entries = 926747 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.54 usrkeys = 926747 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926747 bytes = 3243632 } seq expand size = 6487229 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5125 value = 3.4828 dict = 1.00 all = 3.6100 } Zip/UnZip{ index = 0.0951 value = 0.2871 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 2253.390863 GB avg = 0.199 KB (by entry num) total key len = 307.350471 GB avg = 0.027 KB total ukey len = 237.688662 GB avg = 0.015 KB total ukey num = 15.755357888 Billion total entry num = 11.296671836 Billion write speed all = 35.804357032 MB/sec (with seq num) write speed all = 34.540753977 MB/sec (without seq num) 2024/11/24-04:47:12.643610 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292455: 926747 keys, 37179278 bytes, temperature: kWarm 2024/11/24-04:47:12.643682 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394832643645, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292455, "file_size": 37179278, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4445561385, "table_properties": {"data_size": 31086432, "index_size": 1763136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948916, "raw_average_key_size": 28, "raw_value_size": 108268906, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926747, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394829, "oldest_key_time": 0, "file_creation_time": 1732394829, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292455, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:47:12.684187 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292456.sst) /dev/shm/ToplingTemp/Topling-272834-YFS3zD 2024/11/24-04:47:13.512869 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292456.sst): first pass time = 0.83's, 161.973'MB/sec 2024/11/24-04:47:14.822503 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292456.sst): old prealloc_size = 147639500, real_size = 31267296 2024/11/24-04:47:14.823090 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292456.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921152 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.71 avg-zval = 31.78 2024/11/24-04:47:14.823840 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292456.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:47:15.637298 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292456.sst): second pass time = 1.31's, 82.873'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.244'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3146.278'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 280.331'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31976.512'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.873'MB/sec zip pipeline throughput = 82.873'MB/sec entries = 921152 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.71 avg-zval = 31.78 usrkeys = 921152 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.71 avg-zval = 31.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921152 bytes = 3108896 } seq expand size = 6448064 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4456 value = 3.7039 dict = 1.00 all = 3.8094 } Zip/UnZip{ index = 0.0957 value = 0.2700 dict = 1.00 all = 0.2625 } ---------------------------- total value len = 2253.505736 GB avg = 0.199 KB (by entry num) total key len = 307.376263 GB avg = 0.027 KB total ukey len = 237.707085 GB avg = 0.015 KB total ukey num = 15.756279040 Billion total entry num = 11.297592988 Billion write speed all = 35.804802186 MB/sec (with seq num) write speed all = 34.541149796 MB/sec (without seq num) 2024/11/24-04:47:15.680575 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292456: 921152 keys, 35233492 bytes, temperature: kWarm 2024/11/24-04:47:15.680650 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394835680613, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292456, "file_size": 35233492, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4341701556, "largest_seqno": 4445560196, "table_properties": {"data_size": 29273104, "index_size": 1763712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25792256, "raw_average_key_size": 28, "raw_value_size": 108425604, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921152, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394832, "oldest_key_time": 0, "file_creation_time": 1732394832, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292456, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:47:15.718160 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292457.sst) /dev/shm/ToplingTemp/Topling-272834-w9B6eB 2024/11/24-04:47:16.735478 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292457.sst): first pass time = 1.02's, 131.938'MB/sec 2024/11/24-04:47:18.070668 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292457.sst): old prealloc_size = 147639500, real_size = 32965424 2024/11/24-04:47:18.071257 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292457.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929652 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.37 avg-zval = 33.31 2024/11/24-04:47:18.072037 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292457.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:47:18.931935 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292457.sst): second pass time = 1.33's, 81.104'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.044'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3128.125'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 288.834'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 37028.105'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.104'MB/sec zip pipeline throughput = 81.104'MB/sec entries = 929652 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.37 avg-zval = 33.31 usrkeys = 929652 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.37 avg-zval = 33.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929652 bytes = 3253792 } seq expand size = 6507564 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5138 value = 3.4939 dict = 1.00 all = 3.6205 } Zip/UnZip{ index = 0.0951 value = 0.2862 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 2253.620431 GB avg = 0.199 KB (by entry num) total key len = 307.402293 GB avg = 0.027 KB total ukey len = 237.725678 GB avg = 0.015 KB total ukey num = 15.757208692 Billion total entry num = 11.298522640 Billion write speed all = 35.805120409 MB/sec (with seq num) write speed all = 34.541422248 MB/sec (without seq num) 2024/11/24-04:47:18.979060 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292457: 929652 keys, 37071692 bytes, temperature: kWarm 2024/11/24-04:47:18.979133 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394838979096, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292457, "file_size": 37071692, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4445562613, "table_properties": {"data_size": 30964384, "index_size": 1768448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26030256, "raw_average_key_size": 28, "raw_value_size": 108187568, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929652, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394835, "oldest_key_time": 0, "file_creation_time": 1732394835, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292457, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:47:19.013880 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292458.sst) /dev/shm/ToplingTemp/Topling-272834-vi47EE 2024/11/24-04:47:20.007973 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292458.sst): first pass time = 0.99's, 135.021'MB/sec 2024/11/24-04:47:21.309666 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292458.sst): old prealloc_size = 147639500, real_size = 31336208 2024/11/24-04:47:21.310235 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292458.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.92 2024/11/24-04:47:21.311008 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292458.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:47:22.117929 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292458.sst): second pass time = 1.30's, 83.388'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.673'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3232.116'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 280.130'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33908.682'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.388'MB/sec zip pipeline throughput = 83.388'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.92 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4447 value = 3.6965 dict = 1.00 all = 3.8024 } Zip/UnZip{ index = 0.0957 value = 0.2705 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 2253.735344 GB avg = 0.199 KB (by entry num) total key len = 307.428034 GB avg = 0.027 KB total ukey len = 237.744064 GB avg = 0.015 KB total ukey num = 15.758127992 Billion total entry num = 11.299441940 Billion write speed all = 35.805491972 MB/sec (with seq num) write speed all = 34.541747281 MB/sec (without seq num) 2024/11/24-04:47:22.161734 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292458: 919300 keys, 35298034 bytes, temperature: kWarm 2024/11/24-04:47:22.161803 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394842161767, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292458, "file_size": 35298034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4329611599, "largest_seqno": 4440963619, "table_properties": {"data_size": 29345856, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394839, "oldest_key_time": 0, "file_creation_time": 1732394839, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292458, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:47:22.197688 139957651584576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292459.sst) /dev/shm/ToplingTemp/Topling-272834-66Vt6E 2024/11/24-04:47:22.330893 139957651584576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292459.sst): first pass time = 0.13's, 237.448'MB/sec 2024/11/24-04:47:22.624254 139957651584576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292459.sst): old prealloc_size = 147639500, real_size = 8336032 2024/11/24-04:47:22.624443 139957651584576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292459.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 221842 prefix = 4 raw-key = 0.0035 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0254 GB zip-val = 0.0079 GB avg-val = 114.53 avg-zval = 35.39 2024/11/24-04:47:22.624660 139957651584576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292459.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.032 GB 2024/11/24-04:47:22.822465 139957651584576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00/292459.sst): second pass time = 0.29's, 86.817'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 46.748'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2628.079'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.332'MB/sec write SST data time = 0.03's, 300.511'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.262'MB, throughput = 8403.655'MB/sec zip my value time = 0.29's, unzip length = 0.025'GB zip my value throughput = 86.817'MB/sec zip pipeline throughput = 86.817'MB/sec entries = 221842 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.53 avg-zval = 35.39 usrkeys = 221842 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.53 avg-zval = 35.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 221842 bytes = 721008 } seq expand size = 1552894 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0044 GB value = 0.0254 GB dict = 0.26 MB all = 0.0316 GB } __ZipSize{ index = 0.0004 GB value = 0.0079 GB dict = 0.26 MB all = 0.0093 GB } UnZip/Zip{ index = 10.3471 value = 3.2361 dict = 1.00 all = 3.4127 } Zip/UnZip{ index = 0.0966 value = 0.3090 dict = 1.00 all = 0.2930 } ---------------------------- total value len = 2253.762305 GB avg = 0.199 KB (by entry num) total key len = 307.434245 GB avg = 0.027 KB total ukey len = 237.748501 GB avg = 0.015 KB total ukey num = 15.758349834 Billion total entry num = 11.299663782 Billion write speed all = 35.805603066 MB/sec (with seq num) write speed all = 34.541846012 MB/sec (without seq num) 2024/11/24-04:47:22.838895 139957651584576 [compaction_job.cc:1948] [default] [JOB 41734] Generated table #292459: 221842 keys, 9265517 bytes, temperature: kWarm 2024/11/24-04:47:22.838948 139957651584576 EVENT_LOG_v1 {"time_micros": 1732394842838920, "cf_name": "default", "job": 41734, "event": "table_file_creation", "file_number": 292459, "file_size": 9265517, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4395757447, "largest_seqno": 4444638579, "table_properties": {"data_size": 7851600, "index_size": 428800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6211576, "raw_average_key_size": 28, "raw_value_size": 25408500, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 221842, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394842, "oldest_key_time": 0, "file_creation_time": 1732394842, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292459, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:47:22.852426 139957651584576 [compaction_job.cc:703] job-41734: subcompact[0], size: 0.753530 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41734/att-00 [292438,292439,292440,292441,292442,292443,292444,292445,292446,292447,292448,292449,292450,292451,292452,292453,292454,292455,292456,292457,292458,292459] 2024/11/24-04:47:22.980965 139957651584576 (Original Log Time 2024/11/24-04:46:09.182075) EVENT_LOG_v1 {"time_micros": 1732394769182030, "job": 41734, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [292428], "files_L3": [292401, 292402, 292403, 292404, 292363, 292364, 292365, 292366, 292367, 292368, 292369, 292370, 292371, 292372, 292373, 292374, 292375, 292376, 292377, 292378, 292379, 292380], "score": 0.660033, "input_data_size": 774941187, "oldest_snapshot_seqno": 4445578685} 2024/11/24-04:47:22.980969 139957651584576 (Original Log Time 2024/11/24-04:47:22.978814) [compaction_job.cc:2022] [default] [JOB 41734] Compacted 1@2 + 22@3 files to L3 => 753529647 bytes 2024/11/24-04:47:22.980970 139957651584576 (Original Log Time 2024/11/24-04:47:22.980892) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 22 0 0 0] max score 0.03, MB/sec: 10.5 rd, 10.2 wr, level 3, files in(1, 22) out(22 +0 blob) MB in(27.1, 712.0 +0.0 blob) out(718.6 +0.0 blob), read-write-amplify(53.9) write-amplify(26.6) OK, records in: 19784108, records dropped: 95437 output_compression: Snappy 2024/11/24-04:47:22.980995 139957651584576 (Original Log Time 2024/11/24-04:47:22.980937) EVENT_LOG_v1 {"time_micros": 1732394842980916, "cf": "default", "job": 41734, "event": "compaction_finished", "compaction_time_micros": 73670357, "compaction_time_cpu_micros": 30583459, "output_level": 3, "num_output_files": 22, "total_output_size": 753529647, "num_input_records": 19784108, "num_output_records": 19688671, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 522138859, "file_range_sync_nanos": 0, "file_fsync_nanos": 14630, "file_prepare_write_nanos": 55917, "lsm_state": [0, 0, 0, 22, 0, 0, 0]} 2024/11/24-04:47:23.113252 139957651584576 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732394769,"executesMs":73936,"compactionJobId":41734,"attempt":0,"compactionInputRawBytes":2863746184,"compactionInputZipBytes":774941187,"compactionOutputRawBytes":0,"compactionOutputZipBytes":753529647,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-ce7d87ea0c1de2554bb15600705189b9-35570adcf60982a9-00"}}, response = 2024/11/24-04:47:23.113321 139957651584576 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41734/att-00: olev 3, work 73.936 s, result 7.838 ms, install 2.202 ms, input{raw 2.667 GiB zip 739.042 MiB}