2024/11/23-14:59:16.563374 139957307483712 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00: fork to child time = 0.022840 sec 2024/11/23-14:59:16.564341 139957307483712 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00 2024/11/23-14:59:16.573018 139957307483712 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00 2024/11/23-14:59:16.574252 139957307483712 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:59:16.574609 139957307483712 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:59:16.627273 139957307483712 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28631/att-00/MANIFEST-206347 2024/11/23-14:59:16.663136 139957307483712 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28631/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 265043, last_sequence is 4047246210, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 264884 2024/11/23-14:59:16.663151 139957307483712 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:59:16.663339 139957307483712 [version_set.cc:5721] Creating manifest 265043 2024/11/23-14:59:16.702275 139957307483712 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:59:16.725337 139957307483712 [compaction_job.cc:2437] [default] [JOB 28631] Compacting 1@2 + 25@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-14:59:16.725366 139957307483712 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21392 Base level 2, inputs: [265021(27MB)], [265039(39MB) 265040(5016KB) 264967(38MB) 264968(34MB) 264969(35MB) 264970(34MB) 264971(34MB) 264972(34MB) 264973(36MB) 264974(34MB) 264975(34MB) 264976(35MB) 264977(36MB) 264978(38MB) 264979(36MB) 264980(35MB) 264981(34MB) 264982(36MB) 264983(33MB) 264984(37MB) 264985(33MB) 264986(33MB) 264987(35MB) 264988(38MB) 264989(40MB)] 2024/11/23-14:59:16.734840 139957307483712 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:59:16.734878 139957307483712 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:59:16.734904 139957307483712 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:59:16.734913 139957307483712 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:59:16.734929 139957307483712 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:59:17.352778 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265044.sst) /dev/shm/ToplingTemp/Topling-84908-23xadw 2024/11/23-14:59:18.311237 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265044.sst): first pass time = 0.96's, 140.042'MB/sec 2024/11/23-14:59:20.286950 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265044.sst): old prealloc_size = 147639500, real_size = 37253584 2024/11/23-14:59:20.287544 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265044.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926288 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0352 GB avg-val = 116.90 avg-zval = 38.05 2024/11/23-14:59:20.288300 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265044.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:21.230747 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265044.sst): second pass time = 1.97's, 54.833'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 43.569'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3280.564'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.23's, 176.851'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 38512.744'MB/sec zip my value time = 1.97's, unzip length = 0.108'GB zip my value throughput = 54.833'MB/sec zip pipeline throughput = 89.988'MB/sec entries = 926288 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.90 avg-zval = 38.05 usrkeys = 926288 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.90 avg-zval = 38.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926288 bytes = 3010448 } seq expand size = 6484016 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.4455 value = 3.0720 dict = 1.00 all = 3.2642 } Zip/UnZip{ index = 0.0957 value = 0.3255 dict = 1.00 all = 0.3064 } ---------------------------- total value len = 8.963386 GB avg = 0.168 KB (by entry num) total key len = 1.466757 GB avg = 0.028 KB total ukey len = 1.016404 GB avg = 0.015 KB total ukey num = 0.066476723 Billion total entry num = 0.053284452 Billion write speed all = 0.477373567 MB/sec (with seq num) write speed all = 0.457863507 MB/sec (without seq num) 2024/11/23-14:59:21.285686 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265044: 926288 keys, 41118686 bytes, temperature: kWarm 2024/11/23-14:59:21.286014 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345161285926, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265044, "file_size": 41118686, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777888, "largest_seqno": 4047123377, "table_properties": {"data_size": 35248256, "index_size": 1773568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25936064, "raw_average_key_size": 28, "raw_value_size": 108281788, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926288, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345157, "oldest_key_time": 0, "file_creation_time": 1732345157, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265044, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:21.373864 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265045.sst) /dev/shm/ToplingTemp/Topling-84908-C8taMy 2024/11/23-14:59:22.692737 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265045.sst): first pass time = 1.32's, 101.771'MB/sec 2024/11/23-14:59:24.868777 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265045.sst): old prealloc_size = 147639500, real_size = 35881920 2024/11/23-14:59:24.869336 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265045.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928006 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0339 GB avg-val = 116.63 avg-zval = 36.50 2024/11/23-14:59:24.870162 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265045.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:26.043192 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265045.sst): second pass time = 2.18's, 49.755'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.17's, 34.149'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3298.816'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.247'MB/sec write SST data time = 0.23's, 172.281'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36428.446'MB/sec zip my value time = 2.18's, unzip length = 0.108'GB zip my value throughput = 49.755'MB/sec zip pipeline throughput = 92.121'MB/sec entries = 928006 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.63 avg-zval = 36.50 usrkeys = 928006 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.63 avg-zval = 36.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928006 bytes = 3364032 } seq expand size = 6496042 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.4449 value = 3.1953 dict = 1.00 all = 3.3468 } Zip/UnZip{ index = 0.0957 value = 0.3130 dict = 1.00 all = 0.2988 } ---------------------------- total value len = 9.078115 GB avg = 0.167 KB (by entry num) total key len = 1.492741 GB avg = 0.028 KB total ukey len = 1.034964 GB avg = 0.015 KB total ukey num = 0.067404729 Billion total entry num = 0.054212458 Billion write speed all = 0.483707311 MB/sec (with seq num) write speed all = 0.463861834 MB/sec (without seq num) 2024/11/23-14:59:26.093944 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265045: 928006 keys, 40103316 bytes, temperature: kWarm 2024/11/23-14:59:26.094047 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345166093999, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265045, "file_size": 40103316, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4047122715, "table_properties": {"data_size": 33872752, "index_size": 1776960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984168, "raw_average_key_size": 28, "raw_value_size": 108233684, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928006, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345161, "oldest_key_time": 0, "file_creation_time": 1732345161, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265045, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:26.163248 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265046.sst) /dev/shm/ToplingTemp/Topling-84908-sxv7Xz 2024/11/23-14:59:27.174381 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265046.sst): first pass time = 1.01's, 132.748'MB/sec 2024/11/23-14:59:28.703129 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265046.sst): old prealloc_size = 147639500, real_size = 30485232 2024/11/23-14:59:28.703702 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265046.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931352 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0285 GB avg-val = 116.11 avg-zval = 30.57 2024/11/23-14:59:28.704542 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265046.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:29.720679 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265046.sst): second pass time = 1.53's, 70.775'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 34.229'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3227.900'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.262'MB/sec write SST data time = 0.13's, 272.659'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29584.502'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.775'MB/sec zip pipeline throughput = 97.902'MB/sec entries = 931352 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.11 avg-zval = 30.57 usrkeys = 931352 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.11 avg-zval = 30.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931352 bytes = 3492592 } seq expand size = 6519464 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4472 value = 3.7985 dict = 1.00 all = 3.8536 } Zip/UnZip{ index = 0.0957 value = 0.2633 dict = 1.00 all = 0.2595 } ---------------------------- total value len = 9.192775 GB avg = 0.167 KB (by entry num) total key len = 1.518819 GB avg = 0.028 KB total ukey len = 1.053591 GB avg = 0.015 KB total ukey num = 0.068336081 Billion total entry num = 0.055143810 Billion write speed all = 0.490064782 MB/sec (with seq num) write speed all = 0.469881762 MB/sec (without seq num) 2024/11/23-14:59:29.763370 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265046: 931352 keys, 34828948 bytes, temperature: kWarm 2024/11/23-14:59:29.763439 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345169763403, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265046, "file_size": 34828948, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776133, "table_properties": {"data_size": 28469216, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26077856, "raw_average_key_size": 28, "raw_value_size": 108139968, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931352, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345166, "oldest_key_time": 0, "file_creation_time": 1732345166, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265046, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:29.802575 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265047.sst) /dev/shm/ToplingTemp/Topling-84908-zo3akz 2024/11/23-14:59:30.697552 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265047.sst): first pass time = 0.89's, 149.974'MB/sec 2024/11/23-14:59:32.288300 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265047.sst): old prealloc_size = 147639500, real_size = 33179440 2024/11/23-14:59:32.288845 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265047.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928076 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.62 avg-zval = 33.57 2024/11/23-14:59:32.289716 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265047.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:33.322728 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265047.sst): second pass time = 1.59's, 68.103'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 36.266'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3435.875'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.256'MB/sec write SST data time = 0.13's, 295.084'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31464.405'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.103'MB/sec zip pipeline throughput = 68.103'MB/sec entries = 928076 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.62 avg-zval = 33.57 usrkeys = 928076 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.62 avg-zval = 33.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928076 bytes = 3480304 } seq expand size = 6496532 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3773 value = 3.4736 dict = 1.00 all = 3.5779 } Zip/UnZip{ index = 0.0964 value = 0.2879 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 9.307503 GB avg = 0.166 KB (by entry num) total key len = 1.544805 GB avg = 0.028 KB total ukey len = 1.072152 GB avg = 0.015 KB total ukey num = 0.069264157 Billion total entry num = 0.056071886 Billion write speed all = 0.496420780 MB/sec (with seq num) write speed all = 0.475901459 MB/sec (without seq num) 2024/11/23-14:59:33.368801 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265047: 928076 keys, 37513444 bytes, temperature: kWarm 2024/11/23-14:59:33.368922 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345173368836, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265047, "file_size": 37513444, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3380528257, "largest_seqno": 4012776149, "table_properties": {"data_size": 31158560, "index_size": 1788672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986128, "raw_average_key_size": 28, "raw_value_size": 108231736, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928076, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345169, "oldest_key_time": 0, "file_creation_time": 1732345169, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265047, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:33.404703 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265048.sst) /dev/shm/ToplingTemp/Topling-84908-98PUIy 2024/11/23-14:59:34.386706 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265048.sst): first pass time = 0.98's, 136.683'MB/sec 2024/11/23-14:59:35.947587 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265048.sst): old prealloc_size = 147639500, real_size = 32577712 2024/11/23-14:59:35.948278 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265048.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932137 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.99 avg-zval = 32.80 2024/11/23-14:59:35.949161 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265048.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:36.977866 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265048.sst): second pass time = 1.56's, 69.339'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 35.613'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2639.064'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.265'MB/sec write SST data time = 0.15's, 242.056'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34344.527'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.339'MB/sec zip pipeline throughput = 69.339'MB/sec entries = 932137 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.99 avg-zval = 32.80 usrkeys = 932137 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.99 avg-zval = 32.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932137 bytes = 3262496 } seq expand size = 6524959 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5118 value = 3.5366 dict = 1.00 all = 3.6581 } Zip/UnZip{ index = 0.0951 value = 0.2828 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 9.422146 GB avg = 0.165 KB (by entry num) total key len = 1.570905 GB avg = 0.028 KB total ukey len = 1.090795 GB avg = 0.016 KB total ukey num = 0.070196294 Billion total entry num = 0.057004023 Billion write speed all = 0.502774759 MB/sec (with seq num) write speed all = 0.481917814 MB/sec (without seq num) 2024/11/23-14:59:37.023987 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265048: 932137 keys, 36691014 bytes, temperature: kWarm 2024/11/23-14:59:37.024056 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345177024021, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265048, "file_size": 36691014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776176, "table_properties": {"data_size": 30570976, "index_size": 1773504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26099836, "raw_average_key_size": 28, "raw_value_size": 108117954, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932137, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345173, "oldest_key_time": 0, "file_creation_time": 1732345173, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265048, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:37.060022 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265049.sst) /dev/shm/ToplingTemp/Topling-84908-Dp3Xhz 2024/11/23-14:59:38.367100 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265049.sst): first pass time = 1.31's, 102.688'MB/sec 2024/11/23-14:59:39.821213 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265049.sst): old prealloc_size = 147639500, real_size = 32811264 2024/11/23-14:59:39.821933 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265049.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931885 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.03 avg-zval = 33.05 2024/11/23-14:59:39.822797 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265049.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:40.804039 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265049.sst): second pass time = 1.45's, 74.577'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 37.569'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2550.918'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.281'MB/sec write SST data time = 0.15's, 239.741'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 39869.483'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.577'MB/sec zip pipeline throughput = 74.577'MB/sec entries = 931885 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.03 avg-zval = 33.05 usrkeys = 931885 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.03 avg-zval = 33.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931885 bytes = 3261616 } seq expand size = 6523195 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4475 value = 3.5112 dict = 1.00 all = 3.6350 } Zip/UnZip{ index = 0.0957 value = 0.2848 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 9.536794 GB avg = 0.165 KB (by entry num) total key len = 1.596998 GB avg = 0.028 KB total ukey len = 1.109433 GB avg = 0.016 KB total ukey num = 0.071128179 Billion total entry num = 0.057935908 Billion write speed all = 0.509122556 MB/sec (with seq num) write speed all = 0.487928355 MB/sec (without seq num) 2024/11/23-14:59:40.849295 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265049: 931885 keys, 36923424 bytes, temperature: kWarm 2024/11/23-14:59:40.849360 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345180849328, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265049, "file_size": 36923424, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 4012776192, "table_properties": {"data_size": 30794160, "index_size": 1783936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26092780, "raw_average_key_size": 28, "raw_value_size": 108125074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931885, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345177, "oldest_key_time": 0, "file_creation_time": 1732345177, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265049, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:40.885132 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265050.sst) /dev/shm/ToplingTemp/Topling-84908-w1HKgz 2024/11/23-14:59:42.020499 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265050.sst): first pass time = 1.14's, 118.220'MB/sec 2024/11/23-14:59:43.463167 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265050.sst): old prealloc_size = 147639500, real_size = 33712816 2024/11/23-14:59:43.464010 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265050.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1048416 prefix = 4 raw-key = 0.0168 GB zip-key = 0.0020 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1049 GB zip-val = 0.0314 GB avg-val = 100.02 avg-zval = 29.99 2024/11/23-14:59:43.464477 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265050.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0026 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:44.322900 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265050.sst): second pass time = 1.44's, 72.753'MB/sec, value only(78.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.826'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2449.652'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.410'MB/sec write SST data time = 0.22's, 164.178'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.048'MB, throughput = 36739.824'MB/sec zip my value time = 1.44's, unzip length = 0.105'GB zip my value throughput = 72.753'MB/sec zip pipeline throughput = 72.753'MB/sec entries = 1048416 avg-key = 28.00 avg-zkey = 1.91 avg-val = 100.02 avg-zval = 29.99 usrkeys = 1048416 avg-key = 20.00 avg-zkey = 1.91 avg-val = 102.56 avg-zval = 29.99 TagRS{ kinds = RS_Key0_TagN bytes = 163920 } TagArray{ size = 380866 bytes = 1285440 } seq expand size = 2666062 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0210 GB value = 0.1049 GB dict = 1.05 MB all = 0.1342 GB } __ZipSize{ index = 0.0020 GB value = 0.0314 GB dict = 1.05 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4457 value = 3.3350 dict = 1.00 all = 3.7326 } Zip/UnZip{ index = 0.0957 value = 0.2999 dict = 1.00 all = 0.2679 } ---------------------------- total value len = 9.644322 GB avg = 0.164 KB (by entry num) total key len = 1.626354 GB avg = 0.028 KB total ukey len = 1.130401 GB avg = 0.016 KB total ukey num = 0.072176595 Billion total entry num = 0.058984324 Billion write speed all = 0.515299018 MB/sec (with seq num) write speed all = 0.493724756 MB/sec (without seq num) 2024/11/23-14:59:44.367195 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265050: 1048416 keys, 35958665 bytes, temperature: kWarm 2024/11/23-14:59:44.367262 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345184367229, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265050, "file_size": 35958665, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 31443168, "index_size": 2007360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 29355648, "raw_average_key_size": 28, "raw_value_size": 104862081, "raw_average_value_size": 100, "num_data_blocks": 1, "num_entries": 1048416, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345180, "oldest_key_time": 0, "file_creation_time": 1732345180, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265050, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:44.402516 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265051.sst) /dev/shm/ToplingTemp/Topling-84908-aNTkBz 2024/11/23-14:59:45.437390 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265051.sst): first pass time = 1.03's, 129.699'MB/sec 2024/11/23-14:59:47.050446 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265051.sst): old prealloc_size = 147639500, real_size = 36334496 2024/11/23-14:59:47.051208 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265051.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 965439 prefix = 4 raw-key = 0.0154 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1072 GB zip-val = 0.0343 GB avg-val = 111.02 avg-zval = 35.48 2024/11/23-14:59:47.051573 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265051.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:47.815607 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265051.sst): second pass time = 1.61's, 66.522'MB/sec, value only(79.9% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 49.892'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2485.876'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.395'MB/sec write SST data time = 0.15's, 248.999'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.065'MB, throughput = 37118.164'MB/sec zip my value time = 1.61's, unzip length = 0.107'GB zip my value throughput = 66.522'MB/sec zip pipeline throughput = 66.522'MB/sec entries = 965439 avg-key = 28.00 avg-zkey = 1.90 avg-val = 111.02 avg-zval = 35.48 usrkeys = 965439 avg-key = 20.00 avg-zkey = 1.90 avg-val = 112.80 avg-zval = 35.48 TagRS{ kinds = RS_Key0_TagN bytes = 150960 } TagArray{ size = 244586 bytes = 856064 } seq expand size = 1712102 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0193 GB value = 0.1072 GB dict = 1.06 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.06 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5144 value = 3.1289 dict = 1.00 all = 3.5159 } Zip/UnZip{ index = 0.0951 value = 0.3196 dict = 1.00 all = 0.2844 } ---------------------------- total value len = 9.753220 GB avg = 0.163 KB (by entry num) total key len = 1.653386 GB avg = 0.028 KB total ukey len = 1.149710 GB avg = 0.016 KB total ukey num = 0.073142034 Billion total entry num = 0.059949763 Billion write speed all = 0.521430510 MB/sec (with seq num) write speed all = 0.499506628 MB/sec (without seq num) 2024/11/23-14:59:47.862939 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265051: 965439 keys, 38174786 bytes, temperature: kWarm 2024/11/23-14:59:47.863024 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345187862972, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265051, "file_size": 38174786, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3428374816, "table_properties": {"data_size": 34256528, "index_size": 1836416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 27032292, "raw_average_key_size": 28, "raw_value_size": 107185462, "raw_average_value_size": 111, "num_data_blocks": 1, "num_entries": 965439, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345184, "oldest_key_time": 0, "file_creation_time": 1732345184, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265051, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:47.902835 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265052.sst) /dev/shm/ToplingTemp/Topling-84908-4PljHw 2024/11/23-14:59:49.073688 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265052.sst): first pass time = 1.17's, 114.637'MB/sec 2024/11/23-14:59:50.374537 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265052.sst): old prealloc_size = 147639500, real_size = 31059936 2024/11/23-14:59:50.375162 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265052.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0291 GB avg-val = 116.10 avg-zval = 31.19 2024/11/23-14:59:50.375987 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265052.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:51.198613 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265052.sst): second pass time = 1.30's, 83.209'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.964'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2981.415'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.12's, 284.708'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27113.257'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.209'MB/sec zip pipeline throughput = 83.209'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 31.19 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 31.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3492880 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5145 value = 3.7218 dict = 1.00 all = 3.7910 } Zip/UnZip{ index = 0.0951 value = 0.2687 dict = 1.00 all = 0.2638 } ---------------------------- total value len = 9.867878 GB avg = 0.162 KB (by entry num) total key len = 1.679466 GB avg = 0.028 KB total ukey len = 1.168339 GB avg = 0.016 KB total ukey num = 0.074073463 Billion total entry num = 0.060881192 Billion write speed all = 0.527782440 MB/sec (with seq num) write speed all = 0.505521373 MB/sec (without seq num) 2024/11/23-14:59:51.242431 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265052: 931429 keys, 35404016 bytes, temperature: kWarm 2024/11/23-14:59:51.242513 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345191242466, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265052, "file_size": 35404016, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3428374817, "largest_seqno": 4012775885, "table_properties": {"data_size": 29055184, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345187, "oldest_key_time": 0, "file_creation_time": 1732345187, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265052, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:51.281313 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265053.sst) /dev/shm/ToplingTemp/Topling-84908-J7Fuex 2024/11/23-14:59:52.195855 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265053.sst): first pass time = 0.91's, 146.765'MB/sec 2024/11/23-14:59:53.955963 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265053.sst): old prealloc_size = 147639500, real_size = 30840512 2024/11/23-14:59:53.956531 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265053.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0288 GB avg-val = 116.12 avg-zval = 30.95 2024/11/23-14:59:53.957314 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265053.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:54.987907 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265053.sst): second pass time = 1.76's, 61.467'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 33.871'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3401.091'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.24's, 148.509'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 24151.214'MB/sec zip my value time = 1.76's, unzip length = 0.108'GB zip my value throughput = 61.467'MB/sec zip pipeline throughput = 120.239'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.12 avg-zval = 30.95 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.12 avg-zval = 30.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4437 value = 3.7518 dict = 1.00 all = 3.8401 } Zip/UnZip{ index = 0.0958 value = 0.2665 dict = 1.00 all = 0.2604 } ---------------------------- total value len = 9.982538 GB avg = 0.161 KB (by entry num) total key len = 1.705543 GB avg = 0.028 KB total ukey len = 1.186965 GB avg = 0.016 KB total ukey num = 0.075004772 Billion total entry num = 0.061812501 Billion write speed all = 0.534122452 MB/sec (with seq num) write speed all = 0.511524768 MB/sec (without seq num) 2024/11/23-14:59:55.032893 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265053: 931309 keys, 34951504 bytes, temperature: kWarm 2024/11/23-14:59:55.033003 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345195032943, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265053, "file_size": 34951504, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 28823984, "index_size": 1783488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345191, "oldest_key_time": 0, "file_creation_time": 1732345191, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265053, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:55.095308 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265054.sst) /dev/shm/ToplingTemp/Topling-84908-yThG3z 2024/11/23-14:59:55.994879 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265054.sst): first pass time = 0.90's, 149.210'MB/sec 2024/11/23-14:59:57.328773 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265054.sst): old prealloc_size = 147639500, real_size = 31287408 2024/11/23-14:59:57.329348 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265054.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0293 GB avg-val = 116.09 avg-zval = 31.44 2024/11/23-14:59:57.330166 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265054.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:58.155029 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265054.sst): second pass time = 1.33's, 81.142'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.126'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3208.191'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 278.473'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37043.598'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.142'MB/sec zip pipeline throughput = 81.142'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 31.44 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 31.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5122 value = 3.6929 dict = 1.00 all = 3.7668 } Zip/UnZip{ index = 0.0951 value = 0.2708 dict = 1.00 all = 0.2655 } ---------------------------- total value len = 10.097195 GB avg = 0.161 KB (by entry num) total key len = 1.731623 GB avg = 0.028 KB total ukey len = 1.205594 GB avg = 0.016 KB total ukey num = 0.075936234 Billion total entry num = 0.062743963 Billion write speed all = 0.540475682 MB/sec (with seq num) write speed all = 0.517540789 MB/sec (without seq num) 2024/11/23-14:59:58.198398 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265054: 931462 keys, 35631652 bytes, temperature: kWarm 2024/11/23-14:59:58.198467 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345198198432, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265054, "file_size": 35631652, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 29282208, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345195, "oldest_key_time": 0, "file_creation_time": 1732345195, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265054, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:58.237455 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265055.sst) /dev/shm/ToplingTemp/Topling-84908-obgcZz 2024/11/23-14:59:59.176135 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265055.sst): first pass time = 0.94's, 142.991'MB/sec 2024/11/23-15:00:00.450461 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265055.sst): old prealloc_size = 147639500, real_size = 32342288 2024/11/23-15:00:00.451053 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265055.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925050 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 117.09 avg-zval = 32.81 2024/11/23-15:00:00.451824 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265055.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:01.277249 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265055.sst): second pass time = 1.27's, 85.048'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.210'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3079.311'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 289.381'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37411.012'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.048'MB/sec zip pipeline throughput = 85.048'MB/sec entries = 925050 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.09 avg-zval = 32.81 usrkeys = 925050 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.09 avg-zval = 32.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925050 bytes = 3353328 } seq expand size = 6475350 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5146 value = 3.5688 dict = 1.00 all = 3.6720 } Zip/UnZip{ index = 0.0951 value = 0.2802 dict = 1.00 all = 0.2723 } ---------------------------- total value len = 10.211987 GB avg = 0.160 KB (by entry num) total key len = 1.757525 GB avg = 0.028 KB total ukey len = 1.224095 GB avg = 0.016 KB total ukey num = 0.076861284 Billion total entry num = 0.063669013 Billion write speed all = 0.546826141 MB/sec (with seq num) write speed all = 0.523556433 MB/sec (without seq num) 2024/11/23-15:00:01.321393 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265055: 925050 keys, 36551226 bytes, temperature: kWarm 2024/11/23-15:00:01.321463 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345201321427, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265055, "file_size": 36551226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3612404272, "largest_seqno": 4012775947, "table_properties": {"data_size": 30351296, "index_size": 1759552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25901400, "raw_average_key_size": 28, "raw_value_size": 108316352, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925050, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345198, "oldest_key_time": 0, "file_creation_time": 1732345198, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265055, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:01.356806 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265056.sst) /dev/shm/ToplingTemp/Topling-84908-WQWGNy 2024/11/23-15:00:02.408347 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265056.sst): first pass time = 1.05's, 127.643'MB/sec 2024/11/23-15:00:03.747673 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265056.sst): old prealloc_size = 147639500, real_size = 35774144 2024/11/23-15:00:03.748211 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265056.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924506 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 117.18 avg-zval = 36.53 2024/11/23-15:00:03.748999 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265056.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:04.580617 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265056.sst): second pass time = 1.34's, 80.961'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.001'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3396.844'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 317.023'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37822.340'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.961'MB/sec zip pipeline throughput = 80.961'MB/sec entries = 924506 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.18 avg-zval = 36.53 usrkeys = 924506 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.18 avg-zval = 36.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924506 bytes = 3351344 } seq expand size = 6471542 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4435 value = 3.2077 dict = 1.00 all = 3.3570 } Zip/UnZip{ index = 0.0958 value = 0.3117 dict = 1.00 all = 0.2979 } ---------------------------- total value len = 10.326790 GB avg = 0.160 KB (by entry num) total key len = 1.783411 GB avg = 0.028 KB total ukey len = 1.242585 GB avg = 0.016 KB total ukey num = 0.077785790 Billion total entry num = 0.064593519 Billion write speed all = 0.553170040 MB/sec (with seq num) write speed all = 0.529566006 MB/sec (without seq num) 2024/11/23-15:00:04.630193 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265056: 924506 keys, 39981668 bytes, temperature: kWarm 2024/11/23-15:00:04.630264 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345204630227, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265056, "file_size": 39981668, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33772336, "index_size": 1770496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25886168, "raw_average_key_size": 28, "raw_value_size": 108331612, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924506, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345201, "oldest_key_time": 0, "file_creation_time": 1732345201, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265056, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:04.669069 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265057.sst) /dev/shm/ToplingTemp/Topling-84908-4oMsmx 2024/11/23-15:00:05.716313 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265057.sst): first pass time = 1.05's, 128.168'MB/sec 2024/11/23-15:00:07.037038 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265057.sst): old prealloc_size = 147639500, real_size = 33662400 2024/11/23-15:00:07.037597 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265057.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929884 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0316 GB avg-val = 116.34 avg-zval = 34.04 2024/11/23-15:00:07.038395 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265057.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:07.889118 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265057.sst): second pass time = 1.32's, 81.988'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.387'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3306.656'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 291.675'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35160.834'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.988'MB/sec zip pipeline throughput = 81.988'MB/sec entries = 929884 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.34 avg-zval = 34.04 usrkeys = 929884 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.67 avg-zval = 34.04 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 841751 bytes = 3156576 } seq expand size = 5892257 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4457 value = 3.4181 dict = 1.00 all = 3.5488 } Zip/UnZip{ index = 0.0957 value = 0.2926 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 10.440863 GB avg = 0.159 KB (by entry num) total key len = 1.809448 GB avg = 0.028 KB total ukey len = 1.261183 GB avg = 0.016 KB total ukey num = 0.078715674 Billion total entry num = 0.065523403 Billion write speed all = 0.559485437 MB/sec (with seq num) write speed all = 0.535545219 MB/sec (without seq num) 2024/11/23-15:00:07.936148 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265057: 929884 keys, 37820542 bytes, temperature: kWarm 2024/11/23-15:00:07.936218 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345207936183, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265057, "file_size": 37820542, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 31649328, "index_size": 1780416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26036752, "raw_average_key_size": 28, "raw_value_size": 108181028, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929884, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345204, "oldest_key_time": 0, "file_creation_time": 1732345204, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265057, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:07.978239 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265058.sst) /dev/shm/ToplingTemp/Topling-84908-pqB5Wx 2024/11/23-15:00:08.861052 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265058.sst): first pass time = 0.88's, 152.040'MB/sec 2024/11/23-15:00:10.155119 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265058.sst): old prealloc_size = 147639500, real_size = 32676144 2024/11/23-15:00:10.155663 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265058.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929765 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.36 avg-zval = 33.00 2024/11/23-15:00:10.156679 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265058.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:10.998065 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265058.sst): second pass time = 1.29's, 83.655'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.668'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3362.606'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 281.787'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36393.025'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.655'MB/sec zip pipeline throughput = 83.655'MB/sec entries = 929765 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.36 avg-zval = 33.00 usrkeys = 929765 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.64 avg-zval = 33.00 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 834901 bytes = 3130896 } seq expand size = 5844307 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5836 value = 3.5255 dict = 1.00 all = 3.6464 } Zip/UnZip{ index = 0.0945 value = 0.2837 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 10.554892 GB avg = 0.159 KB (by entry num) total key len = 1.835481 GB avg = 0.028 KB total ukey len = 1.279778 GB avg = 0.016 KB total ukey num = 0.079645439 Billion total entry num = 0.066453168 Billion write speed all = 0.565801891 MB/sec (with seq num) write speed all = 0.541525412 MB/sec (without seq num) 2024/11/23-15:00:11.042679 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265058: 929765 keys, 36808432 bytes, temperature: kWarm 2024/11/23-15:00:11.042746 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345211042711, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265058, "file_size": 36808432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 30686496, "index_size": 1756992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26033420, "raw_average_key_size": 28, "raw_value_size": 108184338, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929765, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345207, "oldest_key_time": 0, "file_creation_time": 1732345207, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265058, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:11.077445 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265059.sst) /dev/shm/ToplingTemp/Topling-84908-5x37nz 2024/11/23-15:00:11.943531 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265059.sst): first pass time = 0.87's, 154.978'MB/sec 2024/11/23-15:00:13.255545 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265059.sst): old prealloc_size = 147639500, real_size = 31907024 2024/11/23-15:00:13.256112 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265059.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931692 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0299 GB avg-val = 116.06 avg-zval = 32.09 2024/11/23-15:00:13.257788 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265059.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:14.097897 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265059.sst): second pass time = 1.31's, 82.466'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.899'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3257.826'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 285.003'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36272.363'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.466'MB/sec zip pipeline throughput = 82.466'MB/sec entries = 931692 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 32.09 usrkeys = 931692 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931692 bytes = 3377392 } seq expand size = 6521844 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5136 value = 3.6162 dict = 1.00 all = 3.7143 } Zip/UnZip{ index = 0.0951 value = 0.2765 dict = 1.00 all = 0.2692 } ---------------------------- total value len = 10.669544 GB avg = 0.158 KB (by entry num) total key len = 1.861569 GB avg = 0.028 KB total ukey len = 1.298412 GB avg = 0.016 KB total ukey num = 0.080577131 Billion total entry num = 0.067384860 Billion write speed all = 0.572147726 MB/sec (with seq num) write speed all = 0.547534368 MB/sec (without seq num) 2024/11/23-15:00:14.142157 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265059: 931692 keys, 36135492 bytes, temperature: kWarm 2024/11/23-15:00:14.142230 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345214142192, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265059, "file_size": 36135492, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776228, "table_properties": {"data_size": 29901568, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26087376, "raw_average_key_size": 28, "raw_value_size": 108130408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931692, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345211, "oldest_key_time": 0, "file_creation_time": 1732345211, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265059, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:14.182227 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265060.sst) /dev/shm/ToplingTemp/Topling-84908-IJI7wy 2024/11/23-15:00:15.028287 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265060.sst): first pass time = 0.85's, 158.645'MB/sec 2024/11/23-15:00:16.343093 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265060.sst): old prealloc_size = 147639500, real_size = 33298976 2024/11/23-15:00:16.343626 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265060.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933210 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0313 GB avg-val = 115.82 avg-zval = 33.52 2024/11/23-15:00:16.344416 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265060.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:17.198180 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265060.sst): second pass time = 1.31's, 82.256'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.895'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3391.629'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 297.627'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35615.618'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.256'MB/sec zip pipeline throughput = 82.256'MB/sec entries = 933210 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.82 avg-zval = 33.52 usrkeys = 933210 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.82 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933210 bytes = 3382896 } seq expand size = 6532470 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4448 value = 3.4557 dict = 1.00 all = 3.5760 } Zip/UnZip{ index = 0.0957 value = 0.2894 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 10.784164 GB avg = 0.158 KB (by entry num) total key len = 1.887698 GB avg = 0.028 KB total ukey len = 1.317076 GB avg = 0.016 KB total ukey num = 0.081510341 Billion total entry num = 0.068318070 Billion write speed all = 0.578492235 MB/sec (with seq num) write speed all = 0.553541541 MB/sec (without seq num) 2024/11/23-15:00:17.243276 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265060: 933210 keys, 37532636 bytes, temperature: kWarm 2024/11/23-15:00:17.243348 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345217243311, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265060, "file_size": 37532636, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 4012776268, "table_properties": {"data_size": 31278544, "index_size": 1786944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26129880, "raw_average_key_size": 28, "raw_value_size": 108087848, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933210, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345214, "oldest_key_time": 0, "file_creation_time": 1732345214, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265060, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:17.283669 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265061.sst) /dev/shm/ToplingTemp/Topling-84908-ckGaNy 2024/11/23-15:00:18.270132 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265061.sst): first pass time = 0.99's, 136.065'MB/sec 2024/11/23-15:00:19.566308 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265061.sst): old prealloc_size = 147639500, real_size = 32731504 2024/11/23-15:00:19.566846 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265061.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924561 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.17 avg-zval = 33.24 2024/11/23-15:00:19.567650 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265061.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:20.400735 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265061.sst): second pass time = 1.30's, 83.627'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.409'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3429.399'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.12's, 296.615'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36822.472'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.627'MB/sec zip pipeline throughput = 83.627'MB/sec entries = 924561 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.17 avg-zval = 33.24 usrkeys = 924561 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.17 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924561 bytes = 3467120 } seq expand size = 6471927 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4430 value = 3.5253 dict = 1.00 all = 3.6221 } Zip/UnZip{ index = 0.0958 value = 0.2837 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 10.898967 GB avg = 0.157 KB (by entry num) total key len = 1.913586 GB avg = 0.028 KB total ukey len = 1.335567 GB avg = 0.016 KB total ukey num = 0.082434902 Billion total entry num = 0.069242631 Billion write speed all = 0.584829461 MB/sec (with seq num) write speed all = 0.559544801 MB/sec (without seq num) 2024/11/23-15:00:20.445678 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265061: 924561 keys, 37054804 bytes, temperature: kWarm 2024/11/23-15:00:20.445750 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345220445715, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265061, "file_size": 37054804, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 30729504, "index_size": 1770688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25887708, "raw_average_key_size": 28, "raw_value_size": 108330142, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924561, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345217, "oldest_key_time": 0, "file_creation_time": 1732345217, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265061, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:20.482575 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265062.sst) /dev/shm/ToplingTemp/Topling-84908-02vnMz 2024/11/23-15:00:21.325274 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265062.sst): first pass time = 0.84's, 159.278'MB/sec 2024/11/23-15:00:22.642203 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265062.sst): old prealloc_size = 147639500, real_size = 34497856 2024/11/23-15:00:22.642772 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265062.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.40 avg-zval = 34.96 2024/11/23-15:00:22.643570 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265062.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:23.490843 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265062.sst): second pass time = 1.32's, 82.202'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.627'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3230.582'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 305.353'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34150.779'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.202'MB/sec zip pipeline throughput = 82.202'MB/sec entries = 929462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.40 avg-zval = 34.96 usrkeys = 929462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.40 avg-zval = 34.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929462 bytes = 3369312 } seq expand size = 6506234 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5147 value = 3.3293 dict = 1.00 all = 3.4664 } Zip/UnZip{ index = 0.0951 value = 0.3004 dict = 1.00 all = 0.2885 } ---------------------------- total value len = 11.013666 GB avg = 0.157 KB (by entry num) total key len = 1.939611 GB avg = 0.028 KB total ukey len = 1.354157 GB avg = 0.016 KB total ukey num = 0.083364364 Billion total entry num = 0.070172093 Billion write speed all = 0.591169429 MB/sec (with seq num) write speed all = 0.565548980 MB/sec (without seq num) 2024/11/23-15:00:23.537102 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265062: 929462 keys, 38719684 bytes, temperature: kWarm 2024/11/23-15:00:23.537172 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345223537137, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265062, "file_size": 38719684, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 32497392, "index_size": 1767936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26024936, "raw_average_key_size": 28, "raw_value_size": 108192800, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345220, "oldest_key_time": 0, "file_creation_time": 1732345220, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265062, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:23.576648 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265063.sst) /dev/shm/ToplingTemp/Topling-84908-xmPvHx 2024/11/23-15:00:24.446214 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265063.sst): first pass time = 0.87's, 154.357'MB/sec 2024/11/23-15:00:25.717725 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265063.sst): old prealloc_size = 147639500, real_size = 31314624 2024/11/23-15:00:25.718272 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265063.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929591 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0293 GB avg-val = 116.38 avg-zval = 31.53 2024/11/23-15:00:25.719093 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265063.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:26.545959 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265063.sst): second pass time = 1.27's, 85.167'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.049'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3371.503'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 281.242'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27709.078'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.167'MB/sec zip pipeline throughput = 85.167'MB/sec entries = 929591 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.38 avg-zval = 31.53 usrkeys = 929591 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.38 avg-zval = 31.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929591 bytes = 3485984 } seq expand size = 6507137 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5119 value = 3.6908 dict = 1.00 all = 3.7645 } Zip/UnZip{ index = 0.0951 value = 0.2709 dict = 1.00 all = 0.2656 } ---------------------------- total value len = 11.128362 GB avg = 0.157 KB (by entry num) total key len = 1.965640 GB avg = 0.028 KB total ukey len = 1.372748 GB avg = 0.016 KB total ukey num = 0.084293955 Billion total entry num = 0.071101684 Billion write speed all = 0.597508606 MB/sec (with seq num) write speed all = 0.571552374 MB/sec (without seq num) 2024/11/23-15:00:26.591748 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265063: 929591 keys, 35653356 bytes, temperature: kWarm 2024/11/23-15:00:26.591818 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345226591782, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265063, "file_size": 35653356, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 29313392, "index_size": 1768640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26028548, "raw_average_key_size": 28, "raw_value_size": 108189234, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929591, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345223, "oldest_key_time": 0, "file_creation_time": 1732345223, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265063, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:26.636271 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265064.sst) /dev/shm/ToplingTemp/Topling-84908-C0N11w 2024/11/23-15:00:27.611334 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265064.sst): first pass time = 0.98's, 137.656'MB/sec 2024/11/23-15:00:28.873914 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265064.sst): old prealloc_size = 147639500, real_size = 32206240 2024/11/23-15:00:28.874523 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265064.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923122 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0302 GB avg-val = 117.40 avg-zval = 32.71 2024/11/23-15:00:28.875339 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265064.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:29.695309 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265064.sst): second pass time = 1.26's, 85.921'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.469'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3019.215'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.12's, 294.191'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 29104.325'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.921'MB/sec zip pipeline throughput = 85.921'MB/sec entries = 923122 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.40 avg-zval = 32.71 usrkeys = 923122 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.40 avg-zval = 32.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923122 bytes = 3461728 } seq expand size = 6461854 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.3753 value = 3.5889 dict = 1.00 all = 3.6746 } Zip/UnZip{ index = 0.0964 value = 0.2786 dict = 1.00 all = 0.2721 } ---------------------------- total value len = 11.243194 GB avg = 0.156 KB (by entry num) total key len = 1.991487 GB avg = 0.028 KB total ukey len = 1.391211 GB avg = 0.016 KB total ukey num = 0.085217077 Billion total entry num = 0.072024806 Billion write speed all = 0.603841357 MB/sec (with seq num) write speed all = 0.577551910 MB/sec (without seq num) 2024/11/23-15:00:29.745519 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265064: 923122 keys, 36525670 bytes, temperature: kWarm 2024/11/23-15:00:29.745589 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345229745554, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265064, "file_size": 36525670, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776360, "table_properties": {"data_size": 30195792, "index_size": 1779456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847416, "raw_average_key_size": 28, "raw_value_size": 108370436, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923122, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345226, "oldest_key_time": 0, "file_creation_time": 1732345226, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265064, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:29.783284 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265065.sst) /dev/shm/ToplingTemp/Topling-84908-sPZ9kx 2024/11/23-15:00:30.731063 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265065.sst): first pass time = 0.95's, 141.618'MB/sec 2024/11/23-15:00:32.221631 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265065.sst): old prealloc_size = 147639500, real_size = 36322400 2024/11/23-15:00:32.222196 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265065.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929950 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0343 GB avg-val = 116.33 avg-zval = 36.89 2024/11/23-15:00:32.222434 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265065.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:32.942246 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265065.sst): second pass time = 1.49's, 72.614'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 52.463'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3369.466'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.390'MB/sec write SST data time = 0.12's, 305.238'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36654.523'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.614'MB/sec zip pipeline throughput = 72.614'MB/sec entries = 929950 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.33 avg-zval = 36.89 usrkeys = 929950 avg-key = 20.00 avg-zkey = 1.92 avg-val = 117.48 avg-zval = 36.89 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 153657 bytes = 480192 } seq expand size = 1075599 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4438 value = 3.1531 dict = 1.00 all = 3.5503 } Zip/UnZip{ index = 0.0958 value = 0.3171 dict = 1.00 all = 0.2817 } ---------------------------- total value len = 11.352449 GB avg = 0.156 KB (by entry num) total key len = 2.017526 GB avg = 0.028 KB total ukey len = 1.409810 GB avg = 0.016 KB total ukey num = 0.086147027 Billion total entry num = 0.072954756 Billion write speed all = 0.609923852 MB/sec (with seq num) write speed all = 0.583298912 MB/sec (without seq num) 2024/11/23-15:00:32.991706 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265065: 929950 keys, 37804627 bytes, temperature: kWarm 2024/11/23-15:00:32.991777 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345232991742, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265065, "file_size": 37804627, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012773626, "table_properties": {"data_size": 34308880, "index_size": 1780864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26038600, "raw_average_key_size": 28, "raw_value_size": 108179148, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929950, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345229, "oldest_key_time": 0, "file_creation_time": 1732345229, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265065, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:33.033363 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265066.sst) /dev/shm/ToplingTemp/Topling-84908-lq0Knw 2024/11/23-15:00:34.072228 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265066.sst): first pass time = 1.04's, 129.201'MB/sec 2024/11/23-15:00:35.518754 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265066.sst): old prealloc_size = 147639500, real_size = 37326032 2024/11/23-15:00:35.519334 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265066.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925520 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0353 GB avg-val = 117.02 avg-zval = 38.18 2024/11/23-15:00:35.519803 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265066.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:36.258282 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265066.sst): second pass time = 1.45's, 74.909'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 53.861'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3159.440'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.377'MB/sec write SST data time = 0.12's, 320.151'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34343.387'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.909'MB/sec zip pipeline throughput = 74.909'MB/sec entries = 925520 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.02 avg-zval = 38.18 usrkeys = 925520 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.05 avg-zval = 38.18 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 400220 bytes = 1500832 } seq expand size = 2801540 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.5154 value = 3.0651 dict = 1.00 all = 3.3697 } Zip/UnZip{ index = 0.0951 value = 0.3263 dict = 1.00 all = 0.2968 } ---------------------------- total value len = 11.463554 GB avg = 0.155 KB (by entry num) total key len = 2.043440 GB avg = 0.028 KB total ukey len = 1.428320 GB avg = 0.016 KB total ukey num = 0.087072547 Billion total entry num = 0.073880276 Billion write speed all = 0.616081330 MB/sec (with seq num) write speed all = 0.589122699 MB/sec (without seq num) 2024/11/23-15:00:36.308548 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265066: 925520 keys, 39830257 bytes, temperature: kWarm 2024/11/23-15:00:36.308618 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345236308584, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265066, "file_size": 39830257, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4047122349, "table_properties": {"data_size": 35334144, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25914560, "raw_average_key_size": 28, "raw_value_size": 108303260, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925520, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345233, "oldest_key_time": 0, "file_creation_time": 1732345233, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265066, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:36.343816 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265067.sst) /dev/shm/ToplingTemp/Topling-84908-Arwuqz 2024/11/23-15:00:37.232741 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265067.sst): first pass time = 0.89's, 150.996'MB/sec 2024/11/23-15:00:38.578133 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265067.sst): old prealloc_size = 147639500, real_size = 37374352 2024/11/23-15:00:38.578700 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265067.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925927 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0354 GB avg-val = 116.96 avg-zval = 38.20 2024/11/23-15:00:38.579403 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265067.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:00:39.381599 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265067.sst): second pass time = 1.34's, 80.538'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 51.183'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3278.620'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.342'MB/sec write SST data time = 0.13's, 327.262'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34906.724'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.538'MB/sec zip pipeline throughput = 80.538'MB/sec entries = 925927 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.96 avg-zval = 38.20 usrkeys = 925927 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.96 avg-zval = 38.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925927 bytes = 2893536 } seq expand size = 6481489 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0354 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.4433 value = 3.0617 dict = 1.00 all = 3.2638 } Zip/UnZip{ index = 0.0958 value = 0.3266 dict = 1.00 all = 0.3064 } ---------------------------- total value len = 11.578327 GB avg = 0.155 KB (by entry num) total key len = 2.069366 GB avg = 0.028 KB total ukey len = 1.446839 GB avg = 0.016 KB total ukey num = 0.087998474 Billion total entry num = 0.074806203 Billion write speed all = 0.622410240 MB/sec (with seq num) write speed all = 0.595117630 MB/sec (without seq num) 2024/11/23-15:00:39.432461 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265067: 925927 keys, 41122974 bytes, temperature: kWarm 2024/11/23-15:00:39.432531 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345239432495, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265067, "file_size": 41122974, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4020169872, "largest_seqno": 4047123336, "table_properties": {"data_size": 35369408, "index_size": 1773248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25925956, "raw_average_key_size": 28, "raw_value_size": 108291834, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925927, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345236, "oldest_key_time": 0, "file_creation_time": 1732345236, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265067, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:39.468420 139957307483712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265068.sst) /dev/shm/ToplingTemp/Topling-84908-vf4JHw 2024/11/23-15:00:39.617270 139957307483712 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265068.sst): first pass time = 0.15's, 238.328'MB/sec 2024/11/23-15:00:39.964388 139957307483712 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265068.sst): old prealloc_size = 147639500, real_size = 10452528 2024/11/23-15:00:39.964575 139957307483712 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265068.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 244061 prefix = 4 raw-key = 0.0039 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0286 GB zip-val = 0.0099 GB avg-val = 117.32 avg-zval = 40.66 2024/11/23-15:00:39.964841 139957307483712 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265068.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.035 GB 2024/11/23-15:00:40.175612 139957307483712 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00/265068.sst): second pass time = 0.35's, 82.641'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 54.942'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2808.320'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.345'MB/sec write SST data time = 0.03's, 338.407'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.294'MB, throughput = 10502.898'MB/sec zip my value time = 0.35's, unzip length = 0.029'GB zip my value throughput = 82.641'MB/sec zip pipeline throughput = 82.641'MB/sec entries = 244061 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.32 avg-zval = 40.66 usrkeys = 244061 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.32 avg-zval = 40.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 244061 bytes = 915248 } seq expand size = 1708427 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0049 GB value = 0.0286 GB dict = 0.29 MB all = 0.0355 GB } __ZipSize{ index = 0.0005 GB value = 0.0099 GB dict = 0.29 MB all = 0.0116 GB } UnZip/Zip{ index = 10.4221 value = 2.8855 dict = 1.00 all = 3.0565 } Zip/UnZip{ index = 0.0959 value = 0.3466 dict = 1.00 all = 0.3272 } ---------------------------- total value len = 11.608668 GB avg = 0.155 KB (by entry num) total key len = 2.076200 GB avg = 0.028 KB total ukey len = 1.451720 GB avg = 0.016 KB total ukey num = 0.088242535 Billion total entry num = 0.075050264 Billion write speed all = 0.624083028 MB/sec (with seq num) write speed all = 0.596702365 MB/sec (without seq num) 2024/11/23-15:00:40.195056 139957307483712 [compaction_job.cc:1948] [default] [JOB 28631] Generated table #265068: 244061 keys, 11603757 bytes, temperature: kWarm 2024/11/23-15:00:40.195110 139957307483712 EVENT_LOG_v1 {"time_micros": 1732345240195081, "cf_name": "default", "job": 28631, "event": "table_file_creation", "file_number": 265068, "file_size": 11603757, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3236084454, "largest_seqno": 4047122892, "table_properties": {"data_size": 9922976, "index_size": 468352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6833708, "raw_average_key_size": 28, "raw_value_size": 28632902, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 244061, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345239, "oldest_key_time": 0, "file_creation_time": 1732345239, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265068, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:00:40.209545 139957307483712 [compaction_job.cc:703] job-28631: subcompact[0], size: 0.910445 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28631/att-00 [265044,265045,265046,265047,265048,265049,265050,265051,265052,265053,265054,265055,265056,265057,265058,265059,265060,265061,265062,265063,265064,265065,265066,265067,265068] 2024/11/23-15:00:40.352581 139957307483712 (Original Log Time 2024/11/23-14:59:16.725425) EVENT_LOG_v1 {"time_micros": 1732345156725381, "job": 28631, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [265021], "files_L3": [265039, 265040, 264967, 264968, 264969, 264970, 264971, 264972, 264973, 264974, 264975, 264976, 264977, 264978, 264979, 264980, 264981, 264982, 264983, 264984, 264985, 264986, 264987, 264988, 264989], "score": 0.997776, "input_data_size": 939238593, "oldest_snapshot_seqno": 4047246130} 2024/11/23-15:00:40.352585 139957307483712 (Original Log Time 2024/11/23-15:00:40.350939) [compaction_job.cc:2022] [default] [JOB 28631] Compacted 1@2 + 25@3 files to L3 => 910444580 bytes 2024/11/23-15:00:40.352586 139957307483712 (Original Log Time 2024/11/23-15:00:40.352507) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 25 0 0 0] max score 0.03, MB/sec: 11.3 rd, 10.9 wr, level 3, files in(1, 25) out(25 +0 blob) MB in(27.1, 868.6 +0.0 blob) out(868.3 +0.0 blob), read-write-amplify(65.1) write-amplify(32.1) OK, records in: 22788247, records dropped: 96147 output_compression: Snappy 2024/11/23-15:00:40.352590 139957307483712 (Original Log Time 2024/11/23-15:00:40.352554) EVENT_LOG_v1 {"time_micros": 1732345240352533, "cf": "default", "job": 28631, "event": "compaction_finished", "compaction_time_micros": 83484126, "compaction_time_cpu_micros": 35788095, "output_level": 3, "num_output_files": 25, "total_output_size": 910444580, "num_input_records": 22788247, "num_output_records": 22692100, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 660613202, "file_range_sync_nanos": 0, "file_fsync_nanos": 18781, "file_prepare_write_nanos": 73326, "lsm_state": [0, 0, 0, 25, 0, 0, 0]} 2024/11/23-15:00:40.482045 139957307483712 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732345156,"executesMs":83779,"compactionJobId":28631,"attempt":0,"compactionInputRawBytes":3270346535,"compactionInputZipBytes":939238593,"compactionOutputRawBytes":0,"compactionOutputZipBytes":910444580,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-0696ce2dfb6e96c23ccd4781d3a4fdaa-59a71792b53e1906-00"}}, response = 2024/11/23-15:00:40.482131 139957307483712 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28631/att-00: olev 3, work 83.780 s, result 7.699 ms, install 1.666 ms, input{raw 3.046 GiB zip 895.728 MiB}