2024/11/23-17:12:24.011752 139957634799168 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00: fork to child time = 0.021767 sec 2024/11/23-17:12:24.012901 139957634799168 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00 2024/11/23-17:12:24.022420 139957634799168 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00 2024/11/23-17:12:24.023827 139957634799168 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:12:24.024196 139957634799168 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:12:24.075291 139957634799168 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31616/att-00/MANIFEST-206347 2024/11/23-17:12:24.108693 139957634799168 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31616/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 271592, last_sequence is 4137908343, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 271432 2024/11/23-17:12:24.108712 139957634799168 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:12:24.108931 139957634799168 [version_set.cc:5721] Creating manifest 271592 2024/11/23-17:12:24.143050 139957634799168 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:12:24.165052 139957634799168 [compaction_job.cc:2437] [default] [JOB 31616] Compacting 1@2 + 24@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-17:12:24.165079 139957634799168 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 23659 Base level 2, inputs: [271564(27MB)], [271586(38MB) 271587(3804KB) 271514(39MB) 271515(34MB) 271516(36MB) 271517(35MB) 271518(35MB) 271519(35MB) 271520(36MB) 271521(37MB) 271522(37MB) 271523(35MB) 271524(34MB) 271525(37MB) 271526(33MB) 271527(37MB) 271528(34MB) 271529(34MB) 271530(36MB) 271531(37MB) 271532(37MB) 271533(5516KB) 271484(37MB) 271485(36MB)] 2024/11/23-17:12:24.174286 139957634799168 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:12:24.174329 139957634799168 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:12:24.174358 139957634799168 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:12:24.174367 139957634799168 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:12:24.174382 139957634799168 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:12:24.872005 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271593.sst) /dev/shm/ToplingTemp/Topling-128311-NEzFSv 2024/11/23-17:12:25.449938 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271593.sst): first pass time = 0.58's, 232.255'MB/sec 2024/11/23-17:12:26.972961 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271593.sst): old prealloc_size = 147639500, real_size = 35772768 2024/11/23-17:12:26.973532 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271593.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928260 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0338 GB avg-val = 116.59 avg-zval = 36.39 2024/11/23-17:12:26.974318 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271593.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:12:27.987739 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271593.sst): second pass time = 1.52's, 71.150'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 39.180'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3341.168'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.262'MB/sec write SST data time = 0.13's, 312.816'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36841.559'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.150'MB/sec zip pipeline throughput = 71.150'MB/sec entries = 928260 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.59 avg-zval = 36.39 usrkeys = 928260 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.59 avg-zval = 36.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928260 bytes = 3132896 } seq expand size = 6497820 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.5159 value = 3.2043 dict = 1.00 all = 3.3758 } Zip/UnZip{ index = 0.0951 value = 0.3121 dict = 1.00 all = 0.2962 } ---------------------------- total value len = 8.963344 GB avg = 0.168 KB (by entry num) total key len = 1.466812 GB avg = 0.028 KB total ukey len = 1.016443 GB avg = 0.015 KB total ukey num = 0.066478695 Billion total entry num = 0.053286424 Billion write speed all = 0.349585594 MB/sec (with seq num) write speed all = 0.335297665 MB/sec (without seq num) 2024/11/23-17:12:28.035252 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271593: 928260 keys, 39758760 bytes, temperature: kWarm 2024/11/23-17:12:28.035344 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353148035289, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271593, "file_size": 39758760, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668904, "largest_seqno": 4137723508, "table_properties": {"data_size": 33775056, "index_size": 1765440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25991280, "raw_average_key_size": 28, "raw_value_size": 108226476, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928260, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353144, "oldest_key_time": 0, "file_creation_time": 1732353144, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271593, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:12:28.079990 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271594.sst) /dev/shm/ToplingTemp/Topling-128311-lCyXgs 2024/11/23-17:12:29.079617 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271594.sst): first pass time = 1.00's, 134.275'MB/sec 2024/11/23-17:12:30.834915 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271594.sst): old prealloc_size = 147639500, real_size = 36694208 2024/11/23-17:12:30.835610 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271594.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930751 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0347 GB avg-val = 116.20 avg-zval = 37.26 2024/11/23-17:12:30.836574 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271594.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:12:31.762372 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271594.sst): second pass time = 1.75's, 61.644'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 44.252'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2643.328'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.15's, 264.808'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33839.689'MB/sec zip my value time = 1.75's, unzip length = 0.108'GB zip my value throughput = 61.644'MB/sec zip pipeline throughput = 61.644'MB/sec entries = 930751 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.20 avg-zval = 37.26 usrkeys = 930751 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.20 avg-zval = 37.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930751 bytes = 3490336 } seq expand size = 6515257 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0410 GB } UnZip/Zip{ index = 10.4464 value = 3.1188 dict = 1.00 all = 3.2704 } Zip/UnZip{ index = 0.0957 value = 0.3206 dict = 1.00 all = 0.3058 } ---------------------------- total value len = 9.078016 GB avg = 0.167 KB (by entry num) total key len = 1.492873 GB avg = 0.028 KB total ukey len = 1.035058 GB avg = 0.015 KB total ukey num = 0.067409446 Billion total entry num = 0.054217175 Billion write speed all = 0.354257697 MB/sec (with seq num) write speed all = 0.339722041 MB/sec (without seq num) 2024/11/23-17:12:31.815788 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271594: 930751 keys, 41040568 bytes, temperature: kWarm 2024/11/23-17:12:31.815857 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353151815822, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271594, "file_size": 41040568, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4137723563, "table_properties": {"data_size": 34679408, "index_size": 1781952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26061028, "raw_average_key_size": 28, "raw_value_size": 108156730, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930751, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353148, "oldest_key_time": 0, "file_creation_time": 1732353148, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271594, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:12:31.857741 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271595.sst) /dev/shm/ToplingTemp/Topling-128311-rFblMt 2024/11/23-17:12:33.248564 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271595.sst): first pass time = 1.39's, 96.505'MB/sec 2024/11/23-17:12:35.105043 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271595.sst): old prealloc_size = 147639500, real_size = 34875248 2024/11/23-17:12:35.105714 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271595.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1085667 prefix = 4 raw-key = 0.0174 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1038 GB zip-val = 0.0325 GB avg-val = 95.63 avg-zval = 29.96 2024/11/23-17:12:35.106040 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271595.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:12:35.916316 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271595.sst): second pass time = 1.85's, 55.976'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 44.984'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3206.769'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.460'MB/sec write SST data time = 0.22's, 165.888'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.036'MB, throughput = 33110.248'MB/sec zip my value time = 1.85's, unzip length = 0.104'GB zip my value throughput = 55.976'MB/sec zip pipeline throughput = 126.948'MB/sec entries = 1085667 avg-key = 28.00 avg-zkey = 1.91 avg-val = 95.63 avg-zval = 29.96 usrkeys = 1085667 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.96 avg-zval = 29.96 TagRS{ kinds = RS_Key0_TagN bytes = 169720 } TagArray{ size = 206942 bytes = 672576 } seq expand size = 1448594 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0217 GB value = 0.1038 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0325 GB dict = 1.04 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4475 value = 3.1920 dict = 1.00 all = 3.6779 } Zip/UnZip{ index = 0.0957 value = 0.3133 dict = 1.00 all = 0.2719 } ---------------------------- total value len = 9.183284 GB avg = 0.166 KB (by entry num) total key len = 1.523272 GB avg = 0.028 KB total ukey len = 1.056772 GB avg = 0.015 KB total ukey num = 0.068495113 Billion total entry num = 0.055302842 Billion write speed all = 0.358754286 MB/sec (with seq num) write speed all = 0.343929625 MB/sec (without seq num) 2024/11/23-17:12:35.964316 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271595: 1085667 keys, 36492690 bytes, temperature: kWarm 2024/11/23-17:12:35.964414 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353155964358, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271595, "file_size": 36492690, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 32525280, "index_size": 2078336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30398676, "raw_average_key_size": 28, "raw_value_size": 103819131, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1085667, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353151, "oldest_key_time": 0, "file_creation_time": 1732353151, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271595, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:12:36.021761 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271596.sst) /dev/shm/ToplingTemp/Topling-128311-fH3upr 2024/11/23-17:12:37.043967 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271596.sst): first pass time = 1.02's, 131.310'MB/sec 2024/11/23-17:12:38.644204 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271596.sst): old prealloc_size = 147639500, real_size = 35865616 2024/11/23-17:12:38.644746 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271596.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923424 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0339 GB avg-val = 117.35 avg-zval = 36.67 2024/11/23-17:12:38.645204 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271596.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:12:39.566037 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271596.sst): second pass time = 1.60's, 67.749'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.683'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3404.211'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.23's, 167.646'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35350.354'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.749'MB/sec zip pipeline throughput = 79.938'MB/sec entries = 923424 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.35 avg-zval = 36.67 usrkeys = 923424 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.48 avg-zval = 36.67 TagRS{ kinds = RS_Key0_TagN bytes = 144400 } TagArray{ size = 413746 bytes = 1551568 } seq expand size = 2896222 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4445 value = 3.1997 dict = 1.00 all = 3.4932 } Zip/UnZip{ index = 0.0957 value = 0.3125 dict = 1.00 all = 0.2863 } ---------------------------- total value len = 9.294542 GB avg = 0.165 KB (by entry num) total key len = 1.549128 GB avg = 0.028 KB total ukey len = 1.075240 GB avg = 0.015 KB total ukey num = 0.069418537 Billion total entry num = 0.056226266 Billion write speed all = 0.363304261 MB/sec (with seq num) write speed all = 0.348233907 MB/sec (without seq num) 2024/11/23-17:12:39.616195 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271596: 923424 keys, 38423113 bytes, temperature: kWarm 2024/11/23-17:12:39.616283 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353159616237, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271596, "file_size": 38423113, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3966588799, "table_properties": {"data_size": 33866304, "index_size": 1768256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25855872, "raw_average_key_size": 28, "raw_value_size": 108361956, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353156, "oldest_key_time": 0, "file_creation_time": 1732353156, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271596, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:12:39.678642 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271597.sst) /dev/shm/ToplingTemp/Topling-128311-csXDvt 2024/11/23-17:12:40.686547 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271597.sst): first pass time = 1.01's, 133.172'MB/sec 2024/11/23-17:12:42.216809 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271597.sst): old prealloc_size = 147639500, real_size = 32463168 2024/11/23-17:12:42.217408 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271597.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 116.10 avg-zval = 32.70 2024/11/23-17:12:42.218249 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271597.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:12:43.284566 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271597.sst): second pass time = 1.53's, 70.705'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 34.364'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3066.005'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.23's, 157.156'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35736.559'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.705'MB/sec zip pipeline throughput = 85.331'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 32.70 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 32.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3376448 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5148 value = 3.5503 dict = 1.00 all = 3.6581 } Zip/UnZip{ index = 0.0951 value = 0.2817 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 9.409200 GB avg = 0.165 KB (by entry num) total key len = 1.575208 GB avg = 0.028 KB total ukey len = 1.093869 GB avg = 0.016 KB total ukey num = 0.070349966 Billion total entry num = 0.057157695 Billion write speed all = 0.367973668 MB/sec (with seq num) write speed all = 0.352655572 MB/sec (without seq num) 2024/11/23-17:12:43.330169 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271597: 931429 keys, 36690796 bytes, temperature: kWarm 2024/11/23-17:12:43.330236 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353163330202, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271597, "file_size": 36690796, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775885, "table_properties": {"data_size": 30458480, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353159, "oldest_key_time": 0, "file_creation_time": 1732353159, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271597, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:12:43.369283 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271598.sst) /dev/shm/ToplingTemp/Topling-128311-UTyV7t 2024/11/23-17:12:44.248987 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271598.sst): first pass time = 0.88's, 152.580'MB/sec 2024/11/23-17:12:45.772387 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271598.sst): old prealloc_size = 147639500, real_size = 32238224 2024/11/23-17:12:45.772912 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271598.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 116.12 avg-zval = 32.45 2024/11/23-17:12:45.773699 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271598.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:12:46.849283 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271598.sst): second pass time = 1.52's, 71.051'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 33.755'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3538.618'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.275'MB/sec write SST data time = 0.23's, 159.213'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 26779.784'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.051'MB/sec zip pipeline throughput = 80.080'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.12 avg-zval = 32.45 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.12 avg-zval = 32.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4437 value = 3.5783 dict = 1.00 all = 3.6925 } Zip/UnZip{ index = 0.0958 value = 0.2795 dict = 1.00 all = 0.2708 } ---------------------------- total value len = 9.523860 GB avg = 0.164 KB (by entry num) total key len = 1.601285 GB avg = 0.028 KB total ukey len = 1.112495 GB avg = 0.016 KB total ukey num = 0.071281275 Billion total entry num = 0.058089004 Billion write speed all = 0.372643806 MB/sec (with seq num) write speed all = 0.357077980 MB/sec (without seq num) 2024/11/23-17:12:46.894379 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271598: 931309 keys, 36349088 bytes, temperature: kWarm 2024/11/23-17:12:46.894454 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353166894413, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271598, "file_size": 36349088, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 30221696, "index_size": 1783488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353163, "oldest_key_time": 0, "file_creation_time": 1732353163, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271598, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:12:46.933205 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271599.sst) /dev/shm/ToplingTemp/Topling-128311-sJFLTt 2024/11/23-17:12:47.882359 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271599.sst): first pass time = 0.95's, 141.413'MB/sec 2024/11/23-17:12:49.454741 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271599.sst): old prealloc_size = 147639500, real_size = 32517728 2024/11/23-17:12:49.455307 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271599.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 116.09 avg-zval = 32.76 2024/11/23-17:12:49.456176 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271599.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:12:50.563278 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271599.sst): second pass time = 1.57's, 68.808'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 33.253'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3251.729'MB/sec (index lex order gen) rebuild zvType time = 0.98's, 0.238'MB/sec write SST data time = 0.13's, 285.201'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29513.605'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.808'MB/sec zip pipeline throughput = 68.808'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 32.76 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 32.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5122 value = 3.5440 dict = 1.00 all = 3.6411 } Zip/UnZip{ index = 0.0951 value = 0.2822 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 9.638517 GB avg = 0.163 KB (by entry num) total key len = 1.627366 GB avg = 0.028 KB total ukey len = 1.131124 GB avg = 0.016 KB total ukey num = 0.072212737 Billion total entry num = 0.059020466 Billion write speed all = 0.377310977 MB/sec (with seq num) write speed all = 0.361497519 MB/sec (without seq num) 2024/11/23-17:12:50.612843 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271599: 931462 keys, 36861828 bytes, temperature: kWarm 2024/11/23-17:12:50.612919 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353170612876, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271599, "file_size": 36861828, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 30512528, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353166, "oldest_key_time": 0, "file_creation_time": 1732353166, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271599, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:12:50.652911 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271600.sst) /dev/shm/ToplingTemp/Topling-128311-g84TGu 2024/11/23-17:12:51.546153 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271600.sst): first pass time = 0.89's, 150.264'MB/sec 2024/11/23-17:12:52.914867 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271600.sst): old prealloc_size = 147639500, real_size = 33964288 2024/11/23-17:12:52.915410 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271600.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0320 GB avg-val = 117.04 avg-zval = 34.55 2024/11/23-17:12:52.916221 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271600.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:12:53.757100 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271600.sst): second pass time = 1.37's, 79.178'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.326'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3383.388'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 298.040'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32811.695'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.178'MB/sec zip pipeline throughput = 79.178'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 34.55 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 34.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5133 value = 3.3875 dict = 1.00 all = 3.5159 } Zip/UnZip{ index = 0.0951 value = 0.2952 dict = 1.00 all = 0.2844 } ---------------------------- total value len = 9.753301 GB avg = 0.163 KB (by entry num) total key len = 1.653277 GB avg = 0.028 KB total ukey len = 1.149632 GB avg = 0.016 KB total ukey num = 0.073138142 Billion total entry num = 0.059945871 Billion write speed all = 0.381982220 MB/sec (with seq num) write speed all = 0.365922535 MB/sec (without seq num) 2024/11/23-17:12:53.807556 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271600: 925405 keys, 38174078 bytes, temperature: kWarm 2024/11/23-17:12:53.807630 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353173807593, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271600, "file_size": 38174078, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 31972336, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353170, "oldest_key_time": 0, "file_creation_time": 1732353170, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271600, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:12:53.849140 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271601.sst) /dev/shm/ToplingTemp/Topling-128311-sQxyou 2024/11/23-17:12:54.815747 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271601.sst): first pass time = 0.97's, 138.862'MB/sec 2024/11/23-17:12:56.209131 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271601.sst): old prealloc_size = 147639500, real_size = 35818320 2024/11/23-17:12:56.209686 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271601.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924151 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 117.23 avg-zval = 36.59 2024/11/23-17:12:56.210505 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271601.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:12:57.052971 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271601.sst): second pass time = 1.39's, 77.830'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.433'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3339.125'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 309.398'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31484.954'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.830'MB/sec zip pipeline throughput = 77.830'MB/sec entries = 924151 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.23 avg-zval = 36.59 usrkeys = 924151 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.23 avg-zval = 36.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924151 bytes = 3350064 } seq expand size = 6469057 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4451 value = 3.2037 dict = 1.00 all = 3.3533 } Zip/UnZip{ index = 0.0957 value = 0.3121 dict = 1.00 all = 0.2982 } ---------------------------- total value len = 9.868112 GB avg = 0.162 KB (by entry num) total key len = 1.679153 GB avg = 0.028 KB total ukey len = 1.168115 GB avg = 0.016 KB total ukey num = 0.074062293 Billion total entry num = 0.060870022 Billion write speed all = 0.386650850 MB/sec (with seq num) write speed all = 0.370345381 MB/sec (without seq num) 2024/11/23-17:12:57.106457 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271601: 924151 keys, 40025006 bytes, temperature: kWarm 2024/11/23-17:12:57.106529 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353177106492, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271601, "file_size": 40025006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33817536, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876228, "raw_average_key_size": 28, "raw_value_size": 108341550, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924151, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353173, "oldest_key_time": 0, "file_creation_time": 1732353173, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271601, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:12:57.153358 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271602.sst) /dev/shm/ToplingTemp/Topling-128311-fu5eku 2024/11/23-17:12:58.105010 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271602.sst): first pass time = 0.95's, 141.044'MB/sec 2024/11/23-17:12:59.461867 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271602.sst): old prealloc_size = 147639500, real_size = 35193072 2024/11/23-17:12:59.462439 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271602.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930745 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.20 avg-zval = 35.65 2024/11/23-17:12:59.463170 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271602.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:00.286764 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271602.sst): second pass time = 1.36's, 79.759'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 47.278'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3175.128'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.339'MB/sec write SST data time = 0.14's, 283.276'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 20860.703'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.759'MB/sec zip pipeline throughput = 79.759'MB/sec entries = 930745 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.20 avg-zval = 35.65 usrkeys = 930745 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.83 avg-zval = 35.65 TagRS{ kinds = RS_Key0_TagN bytes = 145520 } TagArray{ size = 747748 bytes = 2804064 } seq expand size = 5234236 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4467 value = 3.2599 dict = 1.00 all = 3.4417 } Zip/UnZip{ index = 0.0957 value = 0.3068 dict = 1.00 all = 0.2906 } ---------------------------- total value len = 9.981503 GB avg = 0.162 KB (by entry num) total key len = 1.705214 GB avg = 0.028 KB total ukey len = 1.186730 GB avg = 0.016 KB total ukey num = 0.074993038 Billion total entry num = 0.061800767 Billion write speed all = 0.391277922 MB/sec (with seq num) write speed all = 0.374724924 MB/sec (without seq num) 2024/11/23-17:13:00.296611 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271602: 930745 keys, 38997816 bytes, temperature: kWarm 2024/11/23-17:13:00.296670 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353180296639, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271602, "file_size": 38997816, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 33178336, "index_size": 1781888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060860, "raw_average_key_size": 28, "raw_value_size": 108156994, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930745, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353177, "oldest_key_time": 0, "file_creation_time": 1732353177, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271602, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:00.334728 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271603.sst) /dev/shm/ToplingTemp/Topling-128311-4aNqQs 2024/11/23-17:13:01.344678 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271603.sst): first pass time = 1.01's, 132.902'MB/sec 2024/11/23-17:13:02.684320 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271603.sst): old prealloc_size = 147639500, real_size = 32730992 2024/11/23-17:13:02.684839 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271603.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.49 avg-zval = 33.08 2024/11/23-17:13:02.685677 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271603.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:03.536301 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271603.sst): second pass time = 1.34's, 80.849'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.509'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3561.180'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 288.385'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27075.083'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.849'MB/sec zip pipeline throughput = 80.849'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 33.08 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 33.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3483408 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5122 value = 3.5211 dict = 1.00 all = 3.6209 } Zip/UnZip{ index = 0.0951 value = 0.2840 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 10.096214 GB avg = 0.161 KB (by entry num) total key len = 1.731223 GB avg = 0.028 KB total ukey len = 1.205308 GB avg = 0.016 KB total ukey num = 0.075921943 Billion total entry num = 0.062729672 Billion write speed all = 0.395946236 MB/sec (with seq num) write speed all = 0.379146263 MB/sec (without seq num) 2024/11/23-17:13:03.583723 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271603: 928905 keys, 37067694 bytes, temperature: kWarm 2024/11/23-17:13:03.583798 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353183583758, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271603, "file_size": 37067694, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3288092236, "largest_seqno": 4012776210, "table_properties": {"data_size": 30731296, "index_size": 1767296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353180, "oldest_key_time": 0, "file_creation_time": 1732353180, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271603, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:03.623066 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271604.sst) /dev/shm/ToplingTemp/Topling-128311-v9hqLt 2024/11/23-17:13:04.634671 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271604.sst): first pass time = 1.01's, 132.683'MB/sec 2024/11/23-17:13:05.947916 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271604.sst): old prealloc_size = 147639500, real_size = 32457184 2024/11/23-17:13:05.948507 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271604.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932071 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 116.00 avg-zval = 32.67 2024/11/23-17:13:05.949305 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271604.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:06.803632 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271604.sst): second pass time = 1.31's, 82.416'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.874'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3119.584'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 286.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32784.893'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.416'MB/sec zip pipeline throughput = 82.416'MB/sec entries = 932071 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.00 avg-zval = 32.67 usrkeys = 932071 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.00 avg-zval = 32.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932071 bytes = 3378768 } seq expand size = 6524497 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5137 value = 3.5506 dict = 1.00 all = 3.6585 } Zip/UnZip{ index = 0.0951 value = 0.2816 dict = 1.00 all = 0.2733 } ---------------------------- total value len = 10.210858 GB avg = 0.160 KB (by entry num) total key len = 1.757321 GB avg = 0.028 KB total ukey len = 1.223950 GB avg = 0.016 KB total ukey num = 0.076854014 Billion total entry num = 0.063661743 Billion write speed all = 0.400614037 MB/sec (with seq num) write speed all = 0.383566307 MB/sec (without seq num) 2024/11/23-17:13:06.849117 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271604: 932071 keys, 36686884 bytes, temperature: kWarm 2024/11/23-17:13:06.849182 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353186849148, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271604, "file_size": 36686884, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30450896, "index_size": 1773056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26097988, "raw_average_key_size": 28, "raw_value_size": 108119850, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932071, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353183, "oldest_key_time": 0, "file_creation_time": 1732353183, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271604, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:06.889642 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271605.sst) /dev/shm/ToplingTemp/Topling-128311-fFxVMv 2024/11/23-17:13:07.924554 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271605.sst): first pass time = 1.03's, 129.695'MB/sec 2024/11/23-17:13:09.259053 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271605.sst): old prealloc_size = 147639500, real_size = 33969328 2024/11/23-17:13:09.259590 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271605.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933761 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0319 GB avg-val = 115.74 avg-zval = 34.21 2024/11/23-17:13:09.260427 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271605.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:10.108039 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271605.sst): second pass time = 1.33's, 81.064'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.140'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3503.872'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 305.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35296.880'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.064'MB/sec zip pipeline throughput = 81.064'MB/sec entries = 933761 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.74 avg-zval = 34.21 usrkeys = 933761 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.74 avg-zval = 34.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933761 bytes = 3501616 } seq expand size = 6536327 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4442 value = 3.3828 dict = 1.00 all = 3.5024 } Zip/UnZip{ index = 0.0957 value = 0.2956 dict = 1.00 all = 0.2855 } ---------------------------- total value len = 10.325467 GB avg = 0.160 KB (by entry num) total key len = 1.783467 GB avg = 0.028 KB total ukey len = 1.242625 GB avg = 0.016 KB total ukey num = 0.077787775 Billion total entry num = 0.064595504 Billion write speed all = 0.405280708 MB/sec (with seq num) write speed all = 0.387984842 MB/sec (without seq num) 2024/11/23-17:13:10.155411 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271605: 933761 keys, 38321410 bytes, temperature: kWarm 2024/11/23-17:13:10.155488 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353190155452, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271605, "file_size": 38321410, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 31947616, "index_size": 1788096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26145308, "raw_average_key_size": 28, "raw_value_size": 108072438, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933761, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353186, "oldest_key_time": 0, "file_creation_time": 1732353186, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271605, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:10.195193 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271606.sst) /dev/shm/ToplingTemp/Topling-128311-aijZHv 2024/11/23-17:13:11.183753 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271606.sst): first pass time = 0.99's, 135.776'MB/sec 2024/11/23-17:13:12.471726 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271606.sst): old prealloc_size = 147639500, real_size = 30592736 2024/11/23-17:13:12.472282 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271606.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928396 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.57 avg-zval = 30.76 2024/11/23-17:13:12.473691 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271606.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:13.295634 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271606.sst): second pass time = 1.29's, 84.134'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.392'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3383.107'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 267.903'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27391.059'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.134'MB/sec zip pipeline throughput = 84.134'MB/sec entries = 928396 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.57 avg-zval = 30.76 usrkeys = 928396 avg-key = 20.00 avg-zkey = 1.94 avg-val = 123.57 avg-zval = 30.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928396 bytes = 3481504 } seq expand size = 6498772 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.3096 value = 3.7894 dict = 1.00 all = 3.8429 } Zip/UnZip{ index = 0.0970 value = 0.2639 dict = 1.00 all = 0.2602 } ---------------------------- total value len = 10.440189 GB avg = 0.159 KB (by entry num) total key len = 1.809462 GB avg = 0.028 KB total ukey len = 1.261193 GB avg = 0.016 KB total ukey num = 0.078716171 Billion total entry num = 0.065523900 Billion write speed all = 0.409946692 MB/sec (with seq num) write speed all = 0.392404114 MB/sec (without seq num) 2024/11/23-17:13:13.340166 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271606: 928396 keys, 34925742 bytes, temperature: kWarm 2024/11/23-17:13:13.340237 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353193340201, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271606, "file_size": 34925742, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3413507257, "largest_seqno": 4012776300, "table_properties": {"data_size": 28559440, "index_size": 1801024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995088, "raw_average_key_size": 28, "raw_value_size": 108222676, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928396, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353190, "oldest_key_time": 0, "file_creation_time": 1732353190, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271606, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:13.376734 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271607.sst) /dev/shm/ToplingTemp/Topling-128311-qMe3Vt 2024/11/23-17:13:14.356807 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271607.sst): first pass time = 0.98's, 136.952'MB/sec 2024/11/23-17:13:15.711851 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271607.sst): old prealloc_size = 147639500, real_size = 35293456 2024/11/23-17:13:15.712388 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271607.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925543 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 117.02 avg-zval = 35.98 2024/11/23-17:13:15.714576 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271607.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:16.556863 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271607.sst): second pass time = 1.35's, 79.973'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.888'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3408.261'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 313.439'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34576.209'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.973'MB/sec zip pipeline throughput = 79.973'MB/sec entries = 925543 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.02 avg-zval = 35.98 usrkeys = 925543 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.02 avg-zval = 35.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925543 bytes = 3470800 } seq expand size = 6478801 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5133 value = 3.2522 dict = 1.00 all = 3.3877 } Zip/UnZip{ index = 0.0951 value = 0.3075 dict = 1.00 all = 0.2952 } ---------------------------- total value len = 10.554970 GB avg = 0.159 KB (by entry num) total key len = 1.835377 GB avg = 0.028 KB total ukey len = 1.279704 GB avg = 0.016 KB total ukey num = 0.079641714 Billion total entry num = 0.066449443 Billion write speed all = 0.414609992 MB/sec (with seq num) write speed all = 0.396821562 MB/sec (without seq num) 2024/11/23-17:13:16.605273 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271607: 925543 keys, 39619418 bytes, temperature: kWarm 2024/11/23-17:13:16.605342 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353196605306, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271607, "file_size": 39619418, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33301184, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915204, "raw_average_key_size": 28, "raw_value_size": 108302578, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925543, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353193, "oldest_key_time": 0, "file_creation_time": 1732353193, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271607, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:16.643422 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271608.sst) /dev/shm/ToplingTemp/Topling-128311-2wrD6s 2024/11/23-17:13:17.565770 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271608.sst): first pass time = 0.92's, 145.525'MB/sec 2024/11/23-17:13:18.863652 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271608.sst): old prealloc_size = 147639500, real_size = 31216080 2024/11/23-17:13:18.864201 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271608.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0292 GB avg-val = 116.15 avg-zval = 31.37 2024/11/23-17:13:18.866359 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271608.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:19.698386 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271608.sst): second pass time = 1.30's, 83.380'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.601'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3336.631'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 274.569'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33170.132'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.380'MB/sec zip pipeline throughput = 83.380'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 31.37 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 31.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5132 value = 3.7022 dict = 1.00 all = 3.7745 } Zip/UnZip{ index = 0.0951 value = 0.2701 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 10.669635 GB avg = 0.158 KB (by entry num) total key len = 1.861447 GB avg = 0.028 KB total ukey len = 1.298325 GB avg = 0.016 KB total ukey num = 0.080572797 Billion total entry num = 0.067380526 Billion write speed all = 0.419275254 MB/sec (with seq num) write speed all = 0.401239469 MB/sec (without seq num) 2024/11/23-17:13:19.741467 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271608: 931083 keys, 35559280 bytes, temperature: kWarm 2024/11/23-17:13:19.741538 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353199741503, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271608, "file_size": 35559280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 29211840, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353196, "oldest_key_time": 0, "file_creation_time": 1732353196, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271608, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:19.777293 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271609.sst) /dev/shm/ToplingTemp/Topling-128311-UPFkZt 2024/11/23-17:13:20.731644 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271609.sst): first pass time = 0.95's, 140.643'MB/sec 2024/11/23-17:13:22.032167 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271609.sst): old prealloc_size = 147639500, real_size = 31166432 2024/11/23-17:13:22.032707 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271609.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925001 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0292 GB avg-val = 117.10 avg-zval = 31.52 2024/11/23-17:13:22.034739 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271609.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:22.854163 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271609.sst): second pass time = 1.30's, 83.338'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.130'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3451.367'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 271.813'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34839.178'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.338'MB/sec zip pipeline throughput = 83.338'MB/sec entries = 925001 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.10 avg-zval = 31.52 usrkeys = 925001 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.95 avg-zval = 31.52 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 905280 bytes = 3281648 } seq expand size = 6336960 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3752 value = 3.7156 dict = 1.00 all = 3.7859 } Zip/UnZip{ index = 0.0964 value = 0.2691 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 10.784290 GB avg = 0.158 KB (by entry num) total key len = 1.887347 GB avg = 0.028 KB total ukey len = 1.316825 GB avg = 0.016 KB total ukey num = 0.081497798 Billion total entry num = 0.068305527 Billion write speed all = 0.423933289 MB/sec (with seq num) write speed all = 0.405651839 MB/sec (without seq num) 2024/11/23-17:13:22.897580 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271609: 925001 keys, 35451694 bytes, temperature: kWarm 2024/11/23-17:13:22.897648 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353202897613, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271609, "file_size": 35451694, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 29151888, "index_size": 1783104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900028, "raw_average_key_size": 28, "raw_value_size": 108317798, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925001, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353199, "oldest_key_time": 0, "file_creation_time": 1732353199, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271609, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:22.934323 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271610.sst) /dev/shm/ToplingTemp/Topling-128311-i1maet 2024/11/23-17:13:23.880079 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271610.sst): first pass time = 0.95's, 141.921'MB/sec 2024/11/23-17:13:25.439356 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271610.sst): old prealloc_size = 147639500, real_size = 36661088 2024/11/23-17:13:25.439902 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271610.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926981 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0347 GB avg-val = 116.79 avg-zval = 37.38 2024/11/23-17:13:25.440109 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271610.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:26.137631 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271610.sst): second pass time = 1.56's, 69.466'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 54.020'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3380.211'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.405'MB/sec write SST data time = 0.13's, 301.390'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33177.710'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.466'MB/sec zip pipeline throughput = 69.466'MB/sec entries = 926981 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.79 avg-zval = 37.38 usrkeys = 926981 avg-key = 20.00 avg-zkey = 1.91 avg-val = 116.94 avg-zval = 37.38 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 19714 bytes = 54224 } seq expand size = 137998 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4446 value = 3.1241 dict = 1.00 all = 3.5584 } Zip/UnZip{ index = 0.0957 value = 0.3201 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 10.892690 GB avg = 0.157 KB (by entry num) total key len = 1.913303 GB avg = 0.028 KB total ukey len = 1.335365 GB avg = 0.016 KB total ukey num = 0.082424779 Billion total entry num = 0.069232508 Billion write speed all = 0.428381144 MB/sec (with seq num) write speed all = 0.409853630 MB/sec (without seq num) 2024/11/23-17:13:26.183699 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271610: 926981 keys, 37719025 bytes, temperature: kWarm 2024/11/23-17:13:26.183767 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353206183732, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271610, "file_size": 37719025, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3254051395, "table_properties": {"data_size": 34654096, "index_size": 1775040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955468, "raw_average_key_size": 28, "raw_value_size": 108262354, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926981, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353202, "oldest_key_time": 0, "file_creation_time": 1732353202, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271610, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:26.229665 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271611.sst) /dev/shm/ToplingTemp/Topling-128311-Uy7Bis 2024/11/23-17:13:27.095184 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271611.sst): first pass time = 0.87's, 155.080'MB/sec 2024/11/23-17:13:28.514618 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271611.sst): old prealloc_size = 147639500, real_size = 36114064 2024/11/23-17:13:28.515213 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271611.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925349 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0341 GB avg-val = 117.05 avg-zval = 36.86 2024/11/23-17:13:28.516541 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271611.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:29.297448 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271611.sst): second pass time = 1.42's, 76.345'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 50.055'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3104.301'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.354'MB/sec write SST data time = 0.13's, 304.632'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34218.542'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.345'MB/sec zip pipeline throughput = 76.345'MB/sec entries = 925349 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.05 avg-zval = 36.86 usrkeys = 925349 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.21 avg-zval = 36.86 TagRS{ kinds = RS_Key0_TagN bytes = 144680 } TagArray{ size = 550744 bytes = 2065312 } seq expand size = 3855208 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4447 value = 3.1752 dict = 1.00 all = 3.4254 } Zip/UnZip{ index = 0.0957 value = 0.3149 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 11.004853 GB avg = 0.157 KB (by entry num) total key len = 1.939212 GB avg = 0.028 KB total ukey len = 1.353872 GB avg = 0.016 KB total ukey num = 0.083350128 Billion total entry num = 0.070157857 Billion write speed all = 0.432954145 MB/sec (with seq num) write speed all = 0.414180980 MB/sec (without seq num) 2024/11/23-17:13:29.347087 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271611: 925349 keys, 39183106 bytes, temperature: kWarm 2024/11/23-17:13:29.347157 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353209347120, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271611, "file_size": 39183106, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4137722880, "table_properties": {"data_size": 34110656, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25909772, "raw_average_key_size": 28, "raw_value_size": 108308002, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925349, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353206, "oldest_key_time": 0, "file_creation_time": 1732353206, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271611, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:29.384547 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271612.sst) /dev/shm/ToplingTemp/Topling-128311-jyjFxr 2024/11/23-17:13:30.433941 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271612.sst): first pass time = 1.05's, 127.905'MB/sec 2024/11/23-17:13:31.782714 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271612.sst): old prealloc_size = 147639500, real_size = 35065072 2024/11/23-17:13:31.783244 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271612.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927045 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 116.78 avg-zval = 35.67 2024/11/23-17:13:31.783996 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271612.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:32.621338 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271612.sst): second pass time = 1.35's, 80.336'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.425'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3467.874'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 299.844'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33097.154'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.336'MB/sec zip pipeline throughput = 80.336'MB/sec entries = 927045 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 35.67 usrkeys = 927045 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.78 avg-zval = 35.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927045 bytes = 3012912 } seq expand size = 6489315 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.5132 value = 3.2737 dict = 1.00 all = 3.4475 } Zip/UnZip{ index = 0.0951 value = 0.3055 dict = 1.00 all = 0.2901 } ---------------------------- total value len = 11.119603 GB avg = 0.156 KB (by entry num) total key len = 1.965170 GB avg = 0.028 KB total ukey len = 1.372413 GB avg = 0.016 KB total ukey num = 0.084277173 Billion total entry num = 0.071084902 Billion write speed all = 0.437611874 MB/sec (with seq num) write speed all = 0.418592761 MB/sec (without seq num) 2024/11/23-17:13:32.668524 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271612: 927045 keys, 38932306 bytes, temperature: kWarm 2024/11/23-17:13:32.668592 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353212668557, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271612, "file_size": 38932306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4072856654, "largest_seqno": 4137723763, "table_properties": {"data_size": 33069536, "index_size": 1763584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957260, "raw_average_key_size": 28, "raw_value_size": 108260530, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927045, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353209, "oldest_key_time": 0, "file_creation_time": 1732353209, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271612, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:32.711269 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271613.sst) /dev/shm/ToplingTemp/Topling-128311-yko9Bv 2024/11/23-17:13:33.852864 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271613.sst): first pass time = 1.14's, 117.574'MB/sec 2024/11/23-17:13:35.182316 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271613.sst): old prealloc_size = 147639500, real_size = 35772448 2024/11/23-17:13:35.182868 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271613.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928838 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0338 GB avg-val = 116.50 avg-zval = 36.36 2024/11/23-17:13:35.183622 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271613.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:36.009431 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271613.sst): second pass time = 1.33's, 81.438'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.932'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3319.026'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 307.909'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35062.171'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.438'MB/sec zip pipeline throughput = 81.438'MB/sec entries = 928838 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.50 avg-zval = 36.36 usrkeys = 928838 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.50 avg-zval = 36.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928838 bytes = 3018736 } seq expand size = 6501866 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5144 value = 3.2040 dict = 1.00 all = 3.3856 } Zip/UnZip{ index = 0.0951 value = 0.3121 dict = 1.00 all = 0.2954 } ---------------------------- total value len = 11.234315 GB avg = 0.156 KB (by entry num) total key len = 1.991177 GB avg = 0.028 KB total ukey len = 1.390990 GB avg = 0.016 KB total ukey num = 0.085206011 Billion total entry num = 0.072013740 Billion write speed all = 0.442268038 MB/sec (with seq num) write speed all = 0.423002593 MB/sec (without seq num) 2024/11/23-17:13:36.057532 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271613: 928838 keys, 39644176 bytes, temperature: kWarm 2024/11/23-17:13:36.057606 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353216057569, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271613, "file_size": 39644176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4079590784, "largest_seqno": 4137723318, "table_properties": {"data_size": 33773264, "index_size": 1766784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007464, "raw_average_key_size": 28, "raw_value_size": 108210296, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928838, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353212, "oldest_key_time": 0, "file_creation_time": 1732353212, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271613, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:36.092755 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271614.sst) /dev/shm/ToplingTemp/Topling-128311-IpXC5t 2024/11/23-17:13:37.022552 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271614.sst): first pass time = 0.93's, 144.358'MB/sec 2024/11/23-17:13:38.339164 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271614.sst): old prealloc_size = 147639500, real_size = 34672720 2024/11/23-17:13:38.339707 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271614.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925452 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 117.03 avg-zval = 35.31 2024/11/23-17:13:38.340441 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271614.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:13:39.176183 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271614.sst): second pass time = 1.32's, 82.309'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.183'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3389.783'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.12's, 309.346'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35173.186'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.309'MB/sec zip pipeline throughput = 82.309'MB/sec entries = 925452 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.03 avg-zval = 35.31 usrkeys = 925452 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.03 avg-zval = 35.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925452 bytes = 3123408 } seq expand size = 6478164 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5123 value = 3.3141 dict = 1.00 all = 3.4722 } Zip/UnZip{ index = 0.0951 value = 0.3017 dict = 1.00 all = 0.2880 } ---------------------------- total value len = 11.349099 GB avg = 0.156 KB (by entry num) total key len = 2.017090 GB avg = 0.028 KB total ukey len = 1.409499 GB avg = 0.016 KB total ukey num = 0.086131463 Billion total entry num = 0.072939192 Billion write speed all = 0.446925662 MB/sec (with seq num) write speed all = 0.427414702 MB/sec (without seq num) 2024/11/23-17:13:39.222672 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271614: 925452 keys, 38655160 bytes, temperature: kWarm 2024/11/23-17:13:39.222744 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353219222708, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271614, "file_size": 38655160, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4050411210, "largest_seqno": 4137723530, "table_properties": {"data_size": 32680448, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912656, "raw_average_key_size": 28, "raw_value_size": 108305132, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925452, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353216, "oldest_key_time": 0, "file_creation_time": 1732353216, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271614, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:39.300310 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271615.sst) /dev/shm/ToplingTemp/Topling-128311-VkLuVr 2024/11/23-17:13:39.491044 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271615.sst): first pass time = 0.19's, 247.770'MB/sec 2024/11/23-17:13:39.939824 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271615.sst): old prealloc_size = 147639500, real_size = 12854496 2024/11/23-17:13:39.940088 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271615.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 329148 prefix = 4 raw-key = 0.0053 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0380 GB zip-val = 0.0121 GB avg-val = 115.55 avg-zval = 36.90 2024/11/23-17:13:39.940381 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271615.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.047 GB 2024/11/23-17:13:40.230024 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00/271615.sst): second pass time = 0.45's, 84.870'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.29's, 49.060'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2566.370'MB/sec (index lex order gen) rebuild zvType time = 0.24's, 0.337'MB/sec write SST data time = 0.05's, 311.747'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.390'MB, throughput = 12460.467'MB/sec zip my value time = 0.45's, unzip length = 0.038'GB zip my value throughput = 84.870'MB/sec zip pipeline throughput = 84.870'MB/sec entries = 329148 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.55 avg-zval = 36.90 usrkeys = 329148 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.55 avg-zval = 36.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 329148 bytes = 1069744 } seq expand size = 2304036 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0066 GB value = 0.0380 GB dict = 0.39 MB all = 0.0472 GB } __ZipSize{ index = 0.0006 GB value = 0.0121 GB dict = 0.39 MB all = 0.0142 GB } UnZip/Zip{ index = 10.4979 value = 3.1315 dict = 1.00 all = 3.3192 } Zip/UnZip{ index = 0.0953 value = 0.3193 dict = 1.00 all = 0.3013 } ---------------------------- total value len = 11.389435 GB avg = 0.155 KB (by entry num) total key len = 2.026306 GB avg = 0.028 KB total ukey len = 1.416082 GB avg = 0.016 KB total ukey num = 0.086460611 Billion total entry num = 0.073268340 Billion write speed all = 0.448566754 MB/sec (with seq num) write speed all = 0.428968438 MB/sec (without seq num) 2024/11/23-17:13:40.251786 139957634799168 [compaction_job.cc:1948] [default] [JOB 31616] Generated table #271615: 329148 keys, 14234963 bytes, temperature: kWarm 2024/11/23-17:13:40.251850 139957634799168 EVENT_LOG_v1 {"time_micros": 1732353220251817, "cf_name": "default", "job": 31616, "event": "table_file_creation", "file_number": 271615, "file_size": 14234963, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4076718630, "largest_seqno": 4136223882, "table_properties": {"data_size": 12144976, "index_size": 627072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9216144, "raw_average_key_size": 28, "raw_value_size": 38032588, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 329148, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353219, "oldest_key_time": 0, "file_creation_time": 1732353219, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271615, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:13:40.266041 139957634799168 [compaction_job.cc:703] job-31616: subcompact[0], size: 0.848815 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31616/att-00 [271593,271594,271595,271596,271597,271598,271599,271600,271601,271602,271603,271604,271605,271606,271607,271608,271609,271610,271611,271612,271613,271614,271615] 2024/11/23-17:13:40.396068 139957634799168 (Original Log Time 2024/11/23-17:12:24.165158) EVENT_LOG_v1 {"time_micros": 1732353144165100, "job": 31616, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [271564], "files_L3": [271586, 271587, 271514, 271515, 271516, 271517, 271518, 271519, 271520, 271521, 271522, 271523, 271524, 271525, 271526, 271527, 271528, 271529, 271530, 271531, 271532, 271533, 271484, 271485], "score": 0.998488, "input_data_size": 877429824, "oldest_snapshot_seqno": 4137908188} 2024/11/23-17:13:40.396072 139957634799168 (Original Log Time 2024/11/23-17:13:40.394485) [compaction_job.cc:2022] [default] [JOB 31616] Compacted 1@2 + 24@3 files to L3 => 848814601 bytes 2024/11/23-17:13:40.396073 139957634799168 (Original Log Time 2024/11/23-17:13:40.395995) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.5 rd, 11.2 wr, level 3, files in(1, 24) out(23 +0 blob) MB in(27.1, 809.7 +0.0 blob) out(809.5 +0.0 blob), read-write-amplify(60.8) write-amplify(29.9) OK, records in: 21018342, records dropped: 108166 output_compression: Snappy 2024/11/23-17:13:40.396077 139957634799168 (Original Log Time 2024/11/23-17:13:40.396043) EVENT_LOG_v1 {"time_micros": 1732353220396020, "cf": "default", "job": 31616, "event": "compaction_finished", "compaction_time_micros": 76100890, "compaction_time_cpu_micros": 31965050, "output_level": 3, "num_output_files": 23, "total_output_size": 848814601, "num_input_records": 21018342, "num_output_records": 20910176, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 611627080, "file_range_sync_nanos": 0, "file_fsync_nanos": 17910, "file_prepare_write_nanos": 70024, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-17:13:40.535689 139957634799168 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732353143,"executesMs":76374,"compactionJobId":31616,"attempt":0,"compactionInputRawBytes":3015399777,"compactionInputZipBytes":877429824,"compactionOutputRawBytes":0,"compactionOutputZipBytes":848814601,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-91c9fd3e05d08345f6caa7602bd8ba53-3d7aded2232b67ec-00"}}, response = 2024/11/23-17:13:40.535772 139957634799168 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31616/att-00: olev 3, work 76.375 s, result 7.689 ms, install 1.607 ms, input{raw 2.808 GiB zip 836.782 MiB}