2024/11/24-08:56:16.038579 139950401709632 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00: fork to child time = 0.006911 sec 2024/11/24-08:56:16.038966 139950401709632 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00 2024/11/24-08:56:16.048209 139950401709632 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00 2024/11/24-08:56:16.049096 139950401709632 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-08:56:16.049330 139950401709632 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-08:56:16.137380 139950401709632 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-44250/att-00/MANIFEST-206347 2024/11/24-08:56:16.170586 139950401709632 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-44250/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 298215, last_sequence is 4522431499, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 298009 2024/11/24-08:56:16.170598 139950401709632 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-08:56:16.170706 139950401709632 [version_set.cc:5721] Creating manifest 298215 2024/11/24-08:56:16.224285 139950401709632 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-08:56:16.257120 139950401709632 [compaction_job.cc:2437] [default] [JOB 44250] Compacting 1@2 + 20@3 files to L3, score 0.94, subcompactions 1 : 1 2024/11/24-08:56:16.257150 139950401709632 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 33167 Base level 2, inputs: [298148(27MB)], [298192(36MB) 298193(7220KB) 298116(36MB) 298117(35MB) 298118(35MB) 298119(9437KB) 298081(33MB) 298082(34MB) 298083(30MB) 298084(30MB) 298085(35MB) 298086(33MB) 298087(31MB) 298088(34MB) 298089(33MB) 298090(33MB) 298091(34MB) 298092(33MB) 298093(33MB) 298196(35MB)] 2024/11/24-08:56:16.261811 139950401709632 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-08:56:16.261839 139950401709632 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-08:56:16.261860 139950401709632 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-08:56:16.261865 139950401709632 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-08:56:16.261874 139950401709632 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-08:56:17.136202 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298216.sst) /dev/shm/ToplingTemp/Topling-6382-STuAAO 2024/11/24-08:56:17.692535 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298216.sst): first pass time = 0.56's, 241.275'MB/sec 2024/11/24-08:56:18.997333 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298216.sst): old prealloc_size = 147639500, real_size = 33952336 2024/11/24-08:56:18.997844 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298216.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932434 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0319 GB avg-val = 115.94 avg-zval = 34.25 2024/11/24-08:56:18.998657 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298216.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:19.838930 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298216.sst): second pass time = 1.30's, 82.927'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.232'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3788.304'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.12's, 313.113'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 23052.922'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.927'MB/sec zip pipeline throughput = 82.927'MB/sec entries = 932434 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.94 avg-zval = 34.25 usrkeys = 932434 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.94 avg-zval = 34.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932434 bytes = 3263536 } seq expand size = 6527038 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4458 value = 3.3854 dict = 1.00 all = 3.5259 } Zip/UnZip{ index = 0.0957 value = 0.2954 dict = 1.00 all = 0.2836 } ---------------------------- total value len = 0.114637 GB avg = 0.123 KB (by entry num) total key len = 0.026108 GB avg = 0.028 KB total ukey len = 0.018649 GB avg = 0.020 KB total ukey num = 0.000932434 Billion total entry num = 0.000932434 Billion write speed all = 1.237378087 MB/sec (with seq num) write speed all = 1.171797059 MB/sec (without seq num) 2024/11/24-08:56:19.906608 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298216: 932434 keys, 38066238 bytes, temperature: kWarm 2024/11/24-08:56:19.906699 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409779906642, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298216, "file_size": 38066238, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4310565389, "largest_seqno": 4521847366, "table_properties": {"data_size": 31933760, "index_size": 1785280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26108152, "raw_average_key_size": 28, "raw_value_size": 108109588, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409777, "oldest_key_time": 0, "file_creation_time": 1732409777, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298216, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:19.956722 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298217.sst) /dev/shm/ToplingTemp/Topling-6382-AAGLkO 2024/11/24-08:56:20.975539 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298217.sst): first pass time = 1.02's, 131.745'MB/sec 2024/11/24-08:56:22.251266 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298217.sst): old prealloc_size = 147639500, real_size = 33645040 2024/11/24-08:56:22.251799 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298217.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928495 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0316 GB avg-val = 116.55 avg-zval = 34.08 2024/11/24-08:56:22.252569 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298217.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:23.098697 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298217.sst): second pass time = 1.27's, 84.892'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.545'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3439.564'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.12's, 309.217'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 22348.071'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.892'MB/sec zip pipeline throughput = 84.892'MB/sec entries = 928495 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 34.08 usrkeys = 928495 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 34.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928495 bytes = 3249744 } seq expand size = 6499465 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5125 value = 3.4197 dict = 1.00 all = 3.5556 } Zip/UnZip{ index = 0.0951 value = 0.2924 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 0.229356 GB avg = 0.123 KB (by entry num) total key len = 0.052106 GB avg = 0.028 KB total ukey len = 0.037219 GB avg = 0.020 KB total ukey num = 0.001860929 Billion total entry num = 0.001860929 Billion write speed all = 2.405573303 MB/sec (with seq num) write speed all = 2.278334786 MB/sec (without seq num) 2024/11/24-08:56:23.148795 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298217: 928495 keys, 37748176 bytes, temperature: kWarm 2024/11/24-08:56:23.148862 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409783148827, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298217, "file_size": 37748176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4340431912, "largest_seqno": 4521847237, "table_properties": {"data_size": 31646240, "index_size": 1766464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997860, "raw_average_key_size": 28, "raw_value_size": 108219910, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928495, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409779, "oldest_key_time": 0, "file_creation_time": 1732409779, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298217, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:23.190972 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298218.sst) /dev/shm/ToplingTemp/Topling-6382-X2vxiL 2024/11/24-08:56:24.195291 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298218.sst): first pass time = 1.00's, 133.646'MB/sec 2024/11/24-08:56:25.491511 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298218.sst): old prealloc_size = 147639500, real_size = 35461200 2024/11/24-08:56:25.492046 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298218.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925616 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 117.00 avg-zval = 36.15 2024/11/24-08:56:25.492824 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298218.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:26.322259 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298218.sst): second pass time = 1.30's, 83.612'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.477'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3479.015'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.12's, 323.014'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36635.023'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.612'MB/sec zip pipeline throughput = 83.612'MB/sec entries = 925616 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.00 avg-zval = 36.15 usrkeys = 925616 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.00 avg-zval = 36.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925616 bytes = 3123968 } seq expand size = 6479312 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4458 value = 3.2370 dict = 1.00 all = 3.4031 } Zip/UnZip{ index = 0.0957 value = 0.3089 dict = 1.00 all = 0.2939 } ---------------------------- total value len = 0.344136 GB avg = 0.123 KB (by entry num) total key len = 0.078023 GB avg = 0.028 KB total ukey len = 0.055731 GB avg = 0.020 KB total ukey num = 0.002786545 Billion total entry num = 0.002786545 Billion write speed all = 3.511330555 MB/sec (with seq num) write speed all = 3.325912703 MB/sec (without seq num) 2024/11/24-08:56:26.370588 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298218: 925616 keys, 39440182 bytes, temperature: kWarm 2024/11/24-08:56:26.370658 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409786370621, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298218, "file_size": 39440182, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4417871102, "largest_seqno": 4521847492, "table_properties": {"data_size": 33457408, "index_size": 1772224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25917248, "raw_average_key_size": 28, "raw_value_size": 108300588, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925616, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409783, "oldest_key_time": 0, "file_creation_time": 1732409783, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298218, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:26.425990 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298219.sst) /dev/shm/ToplingTemp/Topling-6382-kMZbXJ 2024/11/24-08:56:27.356702 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298219.sst): first pass time = 0.93's, 144.216'MB/sec 2024/11/24-08:56:28.649142 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298219.sst): old prealloc_size = 147639500, real_size = 33221952 2024/11/24-08:56:28.649655 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298219.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928397 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.57 avg-zval = 33.63 2024/11/24-08:56:28.650412 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298219.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:29.500938 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298219.sst): second pass time = 1.29's, 83.793'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.821'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3592.964'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 308.364'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28728.322'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.793'MB/sec zip pipeline throughput = 83.793'MB/sec entries = 928397 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.57 avg-zval = 33.63 usrkeys = 928397 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.57 avg-zval = 33.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928397 bytes = 3249408 } seq expand size = 6498779 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5148 value = 3.4660 dict = 1.00 all = 3.5959 } Zip/UnZip{ index = 0.0951 value = 0.2885 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 0.458857 GB avg = 0.124 KB (by entry num) total key len = 0.104018 GB avg = 0.028 KB total ukey len = 0.074299 GB avg = 0.020 KB total ukey num = 0.003714942 Billion total entry num = 0.003714942 Billion write speed all = 4.561156591 MB/sec (with seq num) write speed all = 4.320329985 MB/sec (without seq num) 2024/11/24-08:56:29.547038 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298219: 928397 keys, 37324890 bytes, temperature: kWarm 2024/11/24-08:56:29.547103 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409789547069, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298219, "file_size": 37324890, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4521847449, "table_properties": {"data_size": 31223792, "index_size": 1765888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995116, "raw_average_key_size": 28, "raw_value_size": 108222698, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928397, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409786, "oldest_key_time": 0, "file_creation_time": 1732409786, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298219, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:29.594982 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298220.sst) /dev/shm/ToplingTemp/Topling-6382-VH9CeK 2024/11/24-08:56:30.619983 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298220.sst): first pass time = 1.02's, 130.948'MB/sec 2024/11/24-08:56:31.888229 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298220.sst): old prealloc_size = 147639500, real_size = 31337872 2024/11/24-08:56:31.888762 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298220.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.92 2024/11/24-08:56:31.889547 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298220.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:32.692881 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298220.sst): second pass time = 1.27's, 85.637'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 44.013'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3458.654'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.12's, 291.169'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35024.049'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.637'MB/sec zip pipeline throughput = 85.637'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.92 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4439 value = 3.6963 dict = 1.00 all = 3.7899 } Zip/UnZip{ index = 0.0957 value = 0.2705 dict = 1.00 all = 0.2639 } ---------------------------- total value len = 0.573770 GB avg = 0.124 KB (by entry num) total key len = 0.129759 GB avg = 0.028 KB total ukey len = 0.092685 GB avg = 0.020 KB total ukey num = 0.004634242 Billion total entry num = 0.004634242 Billion write speed all = 5.557172687 MB/sec (with seq num) write speed all = 5.264325674 MB/sec (without seq num) 2024/11/24-08:56:32.736762 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298220: 919300 keys, 35414578 bytes, temperature: kWarm 2024/11/24-08:56:32.736828 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409792736793, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298220, "file_size": 35414578, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4331764455, "largest_seqno": 4482401979, "table_properties": {"data_size": 29347392, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409789, "oldest_key_time": 0, "file_creation_time": 1732409789, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298220, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:32.785111 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298221.sst) /dev/shm/ToplingTemp/Topling-6382-Zo9qgN 2024/11/24-08:56:33.820482 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298221.sst): first pass time = 1.04's, 129.638'MB/sec 2024/11/24-08:56:35.113973 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298221.sst): old prealloc_size = 147639500, real_size = 33239920 2024/11/24-08:56:35.114560 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298221.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928795 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.51 avg-zval = 33.62 2024/11/24-08:56:35.115320 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298221.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:35.958981 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298221.sst): second pass time = 1.29's, 83.709'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.200'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3137.934'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.12's, 305.812'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37179.023'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.709'MB/sec zip pipeline throughput = 83.709'MB/sec entries = 928795 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.51 avg-zval = 33.62 usrkeys = 928795 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.51 avg-zval = 33.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928795 bytes = 3250800 } seq expand size = 6501565 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4447 value = 3.4651 dict = 1.00 all = 3.5937 } Zip/UnZip{ index = 0.0957 value = 0.2886 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 0.688483 GB avg = 0.124 KB (by entry num) total key len = 0.155765 GB avg = 0.028 KB total ukey len = 0.111261 GB avg = 0.020 KB total ukey num = 0.005563037 Billion total entry num = 0.005563037 Billion write speed all = 6.500995924 MB/sec (with seq num) write speed all = 6.158297756 MB/sec (without seq num) 2024/11/24-08:56:36.004141 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298221: 928795 keys, 37347828 bytes, temperature: kWarm 2024/11/24-08:56:36.004207 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409796004174, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298221, "file_size": 37347828, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4521846983, "table_properties": {"data_size": 31229024, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26006260, "raw_average_key_size": 28, "raw_value_size": 108211534, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928795, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409792, "oldest_key_time": 0, "file_creation_time": 1732409792, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298221, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:36.054070 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298222.sst) /dev/shm/ToplingTemp/Topling-6382-Ge9RjN 2024/11/24-08:56:36.992835 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298222.sst): first pass time = 0.94's, 142.979'MB/sec 2024/11/24-08:56:38.253568 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298222.sst): old prealloc_size = 147639500, real_size = 30438384 2024/11/24-08:56:38.254090 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298222.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929326 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.42 avg-zval = 30.59 2024/11/24-08:56:38.256522 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298222.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:39.073471 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298222.sst): second pass time = 1.26's, 85.871'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.131'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3537.440'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 273.764'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34857.752'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.871'MB/sec zip pipeline throughput = 85.871'MB/sec entries = 929326 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.42 avg-zval = 30.59 usrkeys = 929326 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.42 avg-zval = 30.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929326 bytes = 3252656 } seq expand size = 6505282 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4447 value = 3.8062 dict = 1.00 all = 3.8856 } Zip/UnZip{ index = 0.0957 value = 0.2627 dict = 1.00 all = 0.2574 } ---------------------------- total value len = 0.803185 GB avg = 0.124 KB (by entry num) total key len = 0.181786 GB avg = 0.028 KB total ukey len = 0.129847 GB avg = 0.020 KB total ukey num = 0.006492363 Billion total entry num = 0.006492363 Billion write speed all = 7.406973009 MB/sec (with seq num) write speed all = 7.016392973 MB/sec (without seq num) 2024/11/24-08:56:39.115381 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298222: 929326 keys, 34542112 bytes, temperature: kWarm 2024/11/24-08:56:39.115451 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409799115413, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298222, "file_size": 34542112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4521846283, "table_properties": {"data_size": 28426336, "index_size": 1779520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26021128, "raw_average_key_size": 28, "raw_value_size": 108196708, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929326, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409796, "oldest_key_time": 0, "file_creation_time": 1732409796, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298222, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:39.158409 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298223.sst) /dev/shm/ToplingTemp/Topling-6382-5fNcwL 2024/11/24-08:56:40.051388 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298223.sst): first pass time = 0.89's, 150.308'MB/sec 2024/11/24-08:56:41.266472 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298223.sst): old prealloc_size = 147639500, real_size = 27853184 2024/11/24-08:56:41.266995 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298223.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0258 GB avg-val = 114.00 avg-zval = 27.30 2024/11/24-08:56:41.269029 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298223.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:42.098466 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298223.sst): second pass time = 1.21's, 88.737'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 38.324'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3608.988'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.335'MB/sec write SST data time = 0.13's, 254.492'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 34160.456'MB/sec zip my value time = 1.21's, unzip length = 0.108'GB zip my value throughput = 88.737'MB/sec zip pipeline throughput = 88.737'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.30 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0258 GB dict = 1.07 MB all = 0.0319 GB } UnZip/Zip{ index = 10.4454 value = 4.1753 dict = 1.00 all = 4.2095 } Zip/UnZip{ index = 0.0957 value = 0.2395 dict = 1.00 all = 0.2376 } ---------------------------- total value len = 0.917554 GB avg = 0.123 KB (by entry num) total key len = 0.208252 GB avg = 0.028 KB total ukey len = 0.148751 GB avg = 0.020 KB total ukey num = 0.007437559 Billion total entry num = 0.007437559 Billion write speed all = 8.277743056 MB/sec (with seq num) write speed all = 7.840252130 MB/sec (without seq num) 2024/11/24-08:56:42.137235 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298223: 945196 keys, 31884400 bytes, temperature: kWarm 2024/11/24-08:56:42.137304 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409802137268, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298223, "file_size": 31884400, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4303740047, "largest_seqno": 4405334581, "table_properties": {"data_size": 25806896, "index_size": 1809792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409799, "oldest_key_time": 0, "file_creation_time": 1732409799, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298223, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:42.178642 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298224.sst) /dev/shm/ToplingTemp/Topling-6382-xvyyKM 2024/11/24-08:56:43.091996 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298224.sst): first pass time = 0.91's, 146.958'MB/sec 2024/11/24-08:56:44.312390 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298224.sst): old prealloc_size = 147639500, real_size = 27756912 2024/11/24-08:56:44.312900 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298224.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.20 2024/11/24-08:56:44.313653 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298224.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:45.142877 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298224.sst): second pass time = 1.22's, 88.354'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 38.276'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3684.394'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.335'MB/sec write SST data time = 0.12's, 256.753'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 37931.517'MB/sec zip my value time = 1.22's, unzip length = 0.108'GB zip my value throughput = 88.354'MB/sec zip pipeline throughput = 88.354'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.20 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.4461 value = 4.1909 dict = 1.00 all = 4.2223 } Zip/UnZip{ index = 0.0957 value = 0.2386 dict = 1.00 all = 0.2368 } ---------------------------- total value len = 1.031922 GB avg = 0.123 KB (by entry num) total key len = 0.234717 GB avg = 0.028 KB total ukey len = 0.167655 GB avg = 0.020 KB total ukey num = 0.008382755 Billion total entry num = 0.008382755 Billion write speed all = 9.109348605 MB/sec (with seq num) write speed all = 8.627055512 MB/sec (without seq num) 2024/11/24-08:56:45.182057 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298224: 945196 keys, 31788144 bytes, temperature: kWarm 2024/11/24-08:56:45.182114 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409805182081, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298224, "file_size": 31788144, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4405334582, "largest_seqno": 4510387940, "table_properties": {"data_size": 25710752, "index_size": 1809664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409802, "oldest_key_time": 0, "file_creation_time": 1732409802, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298224, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:45.265994 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298225.sst) /dev/shm/ToplingTemp/Topling-6382-fZwywN 2024/11/24-08:56:46.135214 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298225.sst): first pass time = 0.87's, 154.418'MB/sec 2024/11/24-08:56:47.381810 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298225.sst): old prealloc_size = 147639500, real_size = 32372160 2024/11/24-08:56:47.382295 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298225.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921995 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0304 GB avg-val = 117.57 avg-zval = 32.96 2024/11/24-08:56:47.383921 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298225.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:48.210011 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298225.sst): second pass time = 1.25's, 87.010'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.022'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3774.369'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.327'MB/sec write SST data time = 0.12's, 294.783'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 23409.319'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 87.010'MB/sec zip pipeline throughput = 87.010'MB/sec entries = 921995 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.57 avg-zval = 32.96 usrkeys = 921995 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.57 avg-zval = 32.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921995 bytes = 3226992 } seq expand size = 6453965 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5112 value = 3.5674 dict = 1.00 all = 3.6815 } Zip/UnZip{ index = 0.0951 value = 0.2803 dict = 1.00 all = 0.2716 } ---------------------------- total value len = 1.146778 GB avg = 0.123 KB (by entry num) total key len = 0.260533 GB avg = 0.028 KB total ukey len = 0.186095 GB avg = 0.020 KB total ukey num = 0.009304750 Billion total entry num = 0.009304750 Billion write speed all = 9.902592380 MB/sec (with seq num) write speed all = 9.378806939 MB/sec (without seq num) 2024/11/24-08:56:48.254146 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298225: 921995 keys, 36457676 bytes, temperature: kWarm 2024/11/24-08:56:48.254212 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409808254177, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298225, "file_size": 36457676, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4521843991, "table_properties": {"data_size": 30387184, "index_size": 1754304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25815860, "raw_average_key_size": 28, "raw_value_size": 108401938, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921995, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409805, "oldest_key_time": 0, "file_creation_time": 1732409805, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298225, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:48.314219 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298226.sst) /dev/shm/ToplingTemp/Topling-6382-sflEfO 2024/11/24-08:56:49.081222 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298226.sst): first pass time = 0.77's, 175.002'MB/sec 2024/11/24-08:56:50.344893 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298226.sst): old prealloc_size = 147639500, real_size = 30960624 2024/11/24-08:56:50.345422 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298226.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924440 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.19 avg-zval = 31.31 2024/11/24-08:56:50.346226 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298226.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:51.152069 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298226.sst): second pass time = 1.26's, 85.787'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.425'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3563.063'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.338'MB/sec write SST data time = 0.12's, 285.807'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 36308.126'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.787'MB/sec zip pipeline throughput = 85.787'MB/sec entries = 924440 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.19 avg-zval = 31.31 usrkeys = 924440 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.19 avg-zval = 31.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924440 bytes = 3235552 } seq expand size = 6471080 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.3752 value = 3.7424 dict = 1.00 all = 3.8292 } Zip/UnZip{ index = 0.0964 value = 0.2672 dict = 1.00 all = 0.2611 } ---------------------------- total value len = 1.261583 GB avg = 0.123 KB (by entry num) total key len = 0.286417 GB avg = 0.028 KB total ukey len = 0.204584 GB avg = 0.020 KB total ukey num = 0.010229190 Billion total entry num = 0.010229190 Billion write speed all = 10.671630878 MB/sec (with seq num) write speed all = 10.107485492 MB/sec (without seq num) 2024/11/24-08:56:51.193258 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298226: 924440 keys, 35050768 bytes, temperature: kWarm 2024/11/24-08:56:51.193324 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409811193290, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298226, "file_size": 35050768, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4372794978, "largest_seqno": 4521847509, "table_properties": {"data_size": 28947296, "index_size": 1782016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25884320, "raw_average_key_size": 28, "raw_value_size": 108333408, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924440, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409808, "oldest_key_time": 0, "file_creation_time": 1732409808, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298226, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:51.234078 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298227.sst) /dev/shm/ToplingTemp/Topling-6382-AShJAM 2024/11/24-08:56:52.045045 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298227.sst): first pass time = 0.81's, 165.510'MB/sec 2024/11/24-08:56:53.256715 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298227.sst): old prealloc_size = 147639500, real_size = 27773632 2024/11/24-08:56:53.257219 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298227.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.22 2024/11/24-08:56:53.258017 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298227.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:54.090310 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298227.sst): second pass time = 1.21's, 88.989'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 38.155'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3729.529'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.334'MB/sec write SST data time = 0.13's, 252.687'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 35899.268'MB/sec zip my value time = 1.21's, unzip length = 0.108'GB zip my value throughput = 88.989'MB/sec zip pipeline throughput = 88.989'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.22 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.4457 value = 4.1882 dict = 1.00 all = 4.2200 } Zip/UnZip{ index = 0.0957 value = 0.2388 dict = 1.00 all = 0.2370 } ---------------------------- total value len = 1.375952 GB avg = 0.123 KB (by entry num) total key len = 0.312883 GB avg = 0.028 KB total ukey len = 0.223488 GB avg = 0.020 KB total ukey num = 0.011174386 Billion total entry num = 0.011174386 Billion write speed all = 11.411371174 MB/sec (with seq num) write speed all = 10.807332920 MB/sec (without seq num) 2024/11/24-08:56:54.128264 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298227: 945196 keys, 31804848 bytes, temperature: kWarm 2024/11/24-08:56:54.128331 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409814128297, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298227, "file_size": 31804848, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4393397093, "largest_seqno": 4497453969, "table_properties": {"data_size": 25727408, "index_size": 1809728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409811, "oldest_key_time": 0, "file_creation_time": 1732409811, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298227, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:54.179407 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298228.sst) /dev/shm/ToplingTemp/Topling-6382-OSHzAK 2024/11/24-08:56:54.984435 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298228.sst): first pass time = 0.80's, 166.734'MB/sec 2024/11/24-08:56:56.288531 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298228.sst): old prealloc_size = 147639500, real_size = 32925264 2024/11/24-08:56:56.289037 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298228.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925209 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.07 avg-zval = 33.43 2024/11/24-08:56:56.289794 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298228.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:56:57.127131 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298228.sst): second pass time = 1.30's, 83.132'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.145'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3611.581'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.12's, 297.631'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 38346.678'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.132'MB/sec zip pipeline throughput = 83.132'MB/sec entries = 925209 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.07 avg-zval = 33.43 usrkeys = 925209 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.07 avg-zval = 33.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925209 bytes = 3238240 } seq expand size = 6476463 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5149 value = 3.5014 dict = 1.00 all = 3.6257 } Zip/UnZip{ index = 0.0951 value = 0.2856 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 1.490740 GB avg = 0.123 KB (by entry num) total key len = 0.338789 GB avg = 0.028 KB total ukey len = 0.241992 GB avg = 0.020 KB total ukey num = 0.012099595 Billion total entry num = 0.012099595 Billion write speed all = 12.113471450 MB/sec (with seq num) write speed all = 11.472571507 MB/sec (without seq num) 2024/11/24-08:56:57.171861 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298228: 925209 keys, 37018456 bytes, temperature: kWarm 2024/11/24-08:56:57.171925 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409817171892, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298228, "file_size": 37018456, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4521844929, "table_properties": {"data_size": 30933952, "index_size": 1759808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25905852, "raw_average_key_size": 28, "raw_value_size": 108311994, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925209, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409814, "oldest_key_time": 0, "file_creation_time": 1732409814, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298228, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:56:57.218235 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298229.sst) /dev/shm/ToplingTemp/Topling-6382-XzMfzL 2024/11/24-08:56:58.052153 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298229.sst): first pass time = 0.83's, 160.956'MB/sec 2024/11/24-08:56:59.292581 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298229.sst): old prealloc_size = 147639500, real_size = 31417664 2024/11/24-08:56:59.293075 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298229.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.01 2024/11/24-08:56:59.294057 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298229.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:57:00.102222 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298229.sst): second pass time = 1.24's, 87.498'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.699'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3805.757'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.12's, 285.683'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36036.069'MB/sec zip my value time = 1.24's, unzip length = 0.108'GB zip my value throughput = 87.498'MB/sec zip pipeline throughput = 87.498'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.01 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4439 value = 3.6863 dict = 1.00 all = 3.7937 } Zip/UnZip{ index = 0.0957 value = 0.2713 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 1.605652 GB avg = 0.123 KB (by entry num) total key len = 0.364529 GB avg = 0.028 KB total ukey len = 0.260378 GB avg = 0.020 KB total ukey num = 0.013018895 Billion total entry num = 0.013018895 Billion write speed all = 12.792750836 MB/sec (with seq num) write speed all = 12.116478207 MB/sec (without seq num) 2024/11/24-08:57:00.143695 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298229: 919300 keys, 35379506 bytes, temperature: kWarm 2024/11/24-08:57:00.143763 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409820143728, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298229, "file_size": 35379506, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4354251370, "largest_seqno": 4475431875, "table_properties": {"data_size": 29427184, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409817, "oldest_key_time": 0, "file_creation_time": 1732409817, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298229, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:57:00.185650 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298230.sst) /dev/shm/ToplingTemp/Topling-6382-9bLZpN 2024/11/24-08:57:01.097109 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298230.sst): first pass time = 0.91's, 147.263'MB/sec 2024/11/24-08:57:02.338994 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298230.sst): old prealloc_size = 147639500, real_size = 29462864 2024/11/24-08:57:02.339560 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298230.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936113 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0274 GB avg-val = 115.38 avg-zval = 29.31 2024/11/24-08:57:02.340330 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298230.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:57:03.164622 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298230.sst): second pass time = 1.24's, 87.020'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 40.540'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3275.344'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.13's, 262.209'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 33713.061'MB/sec zip my value time = 1.24's, unzip length = 0.108'GB zip my value throughput = 87.020'MB/sec zip pipeline throughput = 87.020'MB/sec entries = 936113 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.38 avg-zval = 29.31 usrkeys = 936113 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.38 avg-zval = 29.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936113 bytes = 3159392 } seq expand size = 6552791 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0274 GB dict = 1.08 MB all = 0.0335 GB } UnZip/Zip{ index = 10.4436 value = 3.9367 dict = 1.00 all = 4.0101 } Zip/UnZip{ index = 0.0958 value = 0.2540 dict = 1.00 all = 0.2494 } ---------------------------- total value len = 1.720212 GB avg = 0.123 KB (by entry num) total key len = 0.390740 GB avg = 0.028 KB total ukey len = 0.279100 GB avg = 0.020 KB total ukey num = 0.013955008 Billion total entry num = 0.013955008 Billion write speed all = 13.439558408 MB/sec (with seq num) write speed all = 12.728792317 MB/sec (without seq num) 2024/11/24-08:57:03.209785 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298230: 936113 keys, 33469610 bytes, temperature: kWarm 2024/11/24-08:57:03.209847 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409823209813, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298230, "file_size": 33469610, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4438197664, "largest_seqno": 4521847519, "table_properties": {"data_size": 27435968, "index_size": 1792704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26211164, "raw_average_key_size": 28, "raw_value_size": 108006606, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936113, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409820, "oldest_key_time": 0, "file_creation_time": 1732409820, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298230, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:57:03.251628 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298231.sst) /dev/shm/ToplingTemp/Topling-6382-Np2krL 2024/11/24-08:57:04.133249 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298231.sst): first pass time = 0.88's, 152.246'MB/sec 2024/11/24-08:57:05.409925 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298231.sst): old prealloc_size = 147639500, real_size = 32624848 2024/11/24-08:57:05.410449 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298231.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923069 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0306 GB avg-val = 117.40 avg-zval = 33.19 2024/11/24-08:57:05.411203 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298231.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:57:06.242301 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298231.sst): second pass time = 1.28's, 84.938'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.108'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3482.058'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.12's, 295.895'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33686.644'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.938'MB/sec zip pipeline throughput = 84.938'MB/sec entries = 923069 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.40 avg-zval = 33.19 usrkeys = 923069 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.40 avg-zval = 33.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923069 bytes = 3230752 } seq expand size = 6461483 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5127 value = 3.5372 dict = 1.00 all = 3.6558 } Zip/UnZip{ index = 0.0951 value = 0.2827 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 1.835045 GB avg = 0.123 KB (by entry num) total key len = 0.416586 GB avg = 0.028 KB total ukey len = 0.297562 GB avg = 0.020 KB total ukey num = 0.014878077 Billion total entry num = 0.014878077 Billion write speed all = 14.059714385 MB/sec (with seq num) write speed all = 13.316496758 MB/sec (without seq num) 2024/11/24-08:57:06.291020 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298231: 923069 keys, 36713226 bytes, temperature: kWarm 2024/11/24-08:57:06.291086 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409826291051, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298231, "file_size": 36713226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4521844770, "table_properties": {"data_size": 30637824, "index_size": 1756096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25845932, "raw_average_key_size": 28, "raw_value_size": 108371846, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923069, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409823, "oldest_key_time": 0, "file_creation_time": 1732409823, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298231, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:57:06.331706 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298232.sst) /dev/shm/ToplingTemp/Topling-6382-hNuJYK 2024/11/24-08:57:07.147276 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298232.sst): first pass time = 0.82's, 164.577'MB/sec 2024/11/24-08:57:08.407307 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298232.sst): old prealloc_size = 147639500, real_size = 31337808 2024/11/24-08:57:08.407843 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298232.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.92 2024/11/24-08:57:08.408587 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298232.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:57:09.224052 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298232.sst): second pass time = 1.26's, 86.151'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.221'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3410.777'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 279.470'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35990.646'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 86.151'MB/sec zip pipeline throughput = 86.151'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.92 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4458 value = 3.6963 dict = 1.00 all = 3.8022 } Zip/UnZip{ index = 0.0957 value = 0.2705 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 1.949958 GB avg = 0.123 KB (by entry num) total key len = 0.442327 GB avg = 0.028 KB total ukey len = 0.315948 GB avg = 0.020 KB total ukey num = 0.015797377 Billion total entry num = 0.015797377 Billion write speed all = 14.664941226 MB/sec (with seq num) write speed all = 13.890225243 MB/sec (without seq num) 2024/11/24-08:57:09.267219 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298232: 919300 keys, 35299634 bytes, temperature: kWarm 2024/11/24-08:57:09.267280 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409829267249, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298232, "file_size": 35299634, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4358337153, "largest_seqno": 4471564695, "table_properties": {"data_size": 29347648, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409826, "oldest_key_time": 0, "file_creation_time": 1732409826, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298232, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:57:09.315614 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298233.sst) /dev/shm/ToplingTemp/Topling-6382-P7lHJM 2024/11/24-08:57:10.223932 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298233.sst): first pass time = 0.91's, 147.771'MB/sec 2024/11/24-08:57:11.475169 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298233.sst): old prealloc_size = 147639500, real_size = 29621344 2024/11/24-08:57:11.475729 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298233.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934139 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0276 GB avg-val = 115.68 avg-zval = 29.53 2024/11/24-08:57:11.477017 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298233.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:57:12.289848 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298233.sst): second pass time = 1.25's, 86.429'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 41.129'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3338.708'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.339'MB/sec write SST data time = 0.13's, 266.266'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34199.677'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.429'MB/sec zip pipeline throughput = 86.429'MB/sec entries = 934139 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.68 avg-zval = 29.53 usrkeys = 934139 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.68 avg-zval = 29.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934139 bytes = 3035968 } seq expand size = 6538973 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0276 GB dict = 1.08 MB all = 0.0335 GB } UnZip/Zip{ index = 10.3764 value = 3.9171 dict = 1.00 all = 4.0057 } Zip/UnZip{ index = 0.0964 value = 0.2553 dict = 1.00 all = 0.2496 } ---------------------------- total value len = 2.064559 GB avg = 0.123 KB (by entry num) total key len = 0.468482 GB avg = 0.028 KB total ukey len = 0.334630 GB avg = 0.020 KB total ukey num = 0.016731516 Billion total entry num = 0.016731516 Billion write speed all = 15.241353442 MB/sec (with seq num) write speed all = 14.435962782 MB/sec (without seq num) 2024/11/24-08:57:12.329743 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298233: 934139 keys, 33506420 bytes, temperature: kWarm 2024/11/24-08:57:12.329811 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409832329774, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298233, "file_size": 33506420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4459360698, "largest_seqno": 4521847529, "table_properties": {"data_size": 27587088, "index_size": 1800512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26155892, "raw_average_key_size": 28, "raw_value_size": 108061926, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934139, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409829, "oldest_key_time": 0, "file_creation_time": 1732409829, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298233, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:57:12.372179 139950401709632 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298234.sst) /dev/shm/ToplingTemp/Topling-6382-J7rF0L 2024/11/24-08:57:12.705703 139950401709632 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298234.sst): first pass time = 0.33's, 242.188'MB/sec 2024/11/24-08:57:13.478191 139950401709632 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298234.sst): old prealloc_size = 147639500, real_size = 20540240 2024/11/24-08:57:13.478544 139950401709632 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298234.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 553704 prefix = 4 raw-key = 0.0089 GB zip-key = 0.0011 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0653 GB zip-val = 0.0193 GB avg-val = 117.86 avg-zval = 34.94 2024/11/24-08:57:13.479027 139950401709632 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298234.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0014 GB, waited 0.000 sec, Key+Value = 0.081 GB 2024/11/24-08:57:13.989069 139950401709632 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00/298234.sst): second pass time = 0.77's, 84.556'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.51's, 45.019'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3077.031'MB/sec (index lex order gen) rebuild zvType time = 0.44's, 0.317'MB/sec write SST data time = 0.07's, 311.432'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.655'MB, throughput = 22705.421'MB/sec zip my value time = 0.77's, unzip length = 0.065'GB zip my value throughput = 84.556'MB/sec zip pipeline throughput = 84.556'MB/sec entries = 553704 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.86 avg-zval = 34.94 usrkeys = 553704 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.86 avg-zval = 34.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 553704 bytes = 1937984 } seq expand size = 3875928 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0111 GB value = 0.0653 GB dict = 0.65 MB all = 0.0808 GB } __ZipSize{ index = 0.0011 GB value = 0.0193 GB dict = 0.65 MB all = 0.0230 GB } UnZip/Zip{ index = 10.5059 value = 3.3731 dict = 1.00 all = 3.5117 } Zip/UnZip{ index = 0.0952 value = 0.2965 dict = 1.00 all = 0.2848 } ---------------------------- total value len = 2.133695 GB avg = 0.123 KB (by entry num) total key len = 0.483986 GB avg = 0.028 KB total ukey len = 0.345704 GB avg = 0.020 KB total ukey num = 0.017285220 Billion total entry num = 0.017285220 Billion write speed all = 15.591226820 MB/sec (with seq num) write speed all = 14.767603910 MB/sec (without seq num) 2024/11/24-08:57:14.018822 139950401709632 [compaction_job.cc:1948] [default] [JOB 44250] Generated table #298234: 553704 keys, 22998379 bytes, temperature: kWarm 2024/11/24-08:57:14.018890 139950401709632 EVENT_LOG_v1 {"time_micros": 1732409834018855, "cf_name": "default", "job": 44250, "event": "table_file_creation", "file_number": 298234, "file_size": 22998379, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4521847974, "table_properties": {"data_size": 19347520, "index_size": 1054080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 15503712, "raw_average_key_size": 28, "raw_value_size": 65260620, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 553704, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732409832, "oldest_key_time": 0, "file_creation_time": 1732409832, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 298234, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:57:14.044875 139950401709632 [compaction_job.cc:703] job-44250: subcompact[0], size: 0.661255 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44250/att-00 [298216,298217,298218,298219,298220,298221,298222,298223,298224,298225,298226,298227,298228,298229,298230,298231,298232,298233,298234] 2024/11/24-08:57:14.137742 139950401709632 (Original Log Time 2024/11/24-08:56:16.257215) EVENT_LOG_v1 {"time_micros": 1732409776257168, "job": 44250, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [298148], "files_L3": [298192, 298193, 298116, 298117, 298118, 298119, 298081, 298082, 298083, 298084, 298085, 298086, 298087, 298088, 298089, 298090, 298091, 298092, 298093, 298196], "score": 0.94378, "input_data_size": 687295630, "oldest_snapshot_seqno": 4522431310} 2024/11/24-08:57:14.137746 139950401709632 (Original Log Time 2024/11/24-08:57:14.136261) [compaction_job.cc:2022] [default] [JOB 44250] Compacted 1@2 + 20@3 files to L3 => 661255071 bytes 2024/11/24-08:57:14.137747 139950401709632 (Original Log Time 2024/11/24-08:57:14.137674) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 19 0 0 0] max score 0.02, MB/sec: 11.9 rd, 11.4 wr, level 3, files in(1, 20) out(19 +0 blob) MB in(27.1, 628.4 +0.0 blob) out(630.6 +0.0 blob), read-write-amplify(47.5) write-amplify(23.3) OK, records in: 17332084, records dropped: 46864 output_compression: Snappy 2024/11/24-08:57:14.137751 139950401709632 (Original Log Time 2024/11/24-08:57:14.137717) EVENT_LOG_v1 {"time_micros": 1732409834137697, "cf": "default", "job": 44250, "event": "compaction_finished", "compaction_time_micros": 57787668, "compaction_time_cpu_micros": 25556801, "output_level": 3, "num_output_files": 19, "total_output_size": 661255071, "num_input_records": 17332084, "num_output_records": 17285220, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 436594121, "file_range_sync_nanos": 0, "file_fsync_nanos": 12638, "file_prepare_write_nanos": 37355, "lsm_state": [0, 0, 0, 19, 0, 0, 0]} 2024/11/24-08:57:14.285976 139950401709632 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732409775,"executesMs":58090,"compactionJobId":44250,"attempt":0,"compactionInputRawBytes":2503339444,"compactionInputZipBytes":687295630,"compactionOutputRawBytes":0,"compactionOutputZipBytes":661255071,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5bde425d7c868af55a6e6096483df8b4-c0c0947ea2ab052c-00"}}, response = 2024/11/24-08:57:14.286054 139950401709632 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-44250/att-00: olev 3, work 58.090 s, result 7.147 ms, install 1.504 ms, input{raw 2.331 GiB zip 655.456 MiB}