2024/11/23-14:41:09.592671 139957500515904 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00: fork to child time = 0.008366 sec 2024/11/23-14:41:09.593436 139957500515904 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00 2024/11/23-14:41:09.600733 139957500515904 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00 2024/11/23-14:41:09.601634 139957500515904 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:41:09.601894 139957500515904 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:41:09.658692 139957500515904 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28164/att-00/MANIFEST-206347 2024/11/23-14:41:09.674400 139957500515904 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28164/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 264037, last_sequence is 4033532517, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 263874 2024/11/23-14:41:09.674406 139957500515904 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:41:09.674540 139957500515904 [version_set.cc:5721] Creating manifest 264037 2024/11/23-14:41:09.725129 139957500515904 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:41:09.748013 139957500515904 [compaction_job.cc:2437] [default] [JOB 28164] Compacting 1@2 + 27@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-14:41:09.748053 139957500515904 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21037 Base level 2, inputs: [264034(24MB)], [263950(37MB) 263983(39MB) 263984(38MB) 263985(33MB) 263986(34MB) 263987(35MB) 263988(34MB) 263989(34MB) 263990(34MB) 263991(36MB) 263992(34MB) 263993(34MB) 263994(34MB) 263995(35MB) 263996(36MB) 263997(36MB) 263998(36MB) 263999(35MB) 264000(34MB) 264001(36MB) 264002(34MB) 264003(37MB) 264004(33MB) 264005(33MB) 264006(36MB) 264007(38MB) 264008(40MB)] 2024/11/23-14:41:09.755292 139957500515904 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:41:09.755324 139957500515904 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:41:09.755344 139957500515904 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:41:09.755350 139957500515904 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:41:09.755361 139957500515904 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:41:10.304707 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264038.sst) /dev/shm/ToplingTemp/Topling-78110-tfuq1x 2024/11/23-14:41:11.451350 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264038.sst): first pass time = 1.15's, 117.057'MB/sec 2024/11/23-14:41:13.038767 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264038.sst): old prealloc_size = 147639500, real_size = 35140160 2024/11/23-14:41:13.039455 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264038.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929475 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.40 avg-zval = 35.67 2024/11/23-14:41:13.040312 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264038.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:14.152289 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264038.sst): second pass time = 1.59's, 68.193'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 35.144'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2649.073'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.249'MB/sec write SST data time = 0.18's, 219.906'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36683.974'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.193'MB/sec zip pipeline throughput = 68.193'MB/sec entries = 929475 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.40 avg-zval = 35.67 usrkeys = 929475 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.40 avg-zval = 35.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929475 bytes = 3136992 } seq expand size = 6506325 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5826 value = 3.2636 dict = 1.00 all = 3.4297 } Zip/UnZip{ index = 0.0945 value = 0.3064 dict = 1.00 all = 0.2916 } ---------------------------- total value len = 8.963319 GB avg = 0.168 KB (by entry num) total key len = 1.466846 GB avg = 0.028 KB total ukey len = 1.016468 GB avg = 0.015 KB total ukey num = 0.066479910 Billion total entry num = 0.053287639 Billion write speed all = 0.502369537 MB/sec (with seq num) write speed all = 0.481836718 MB/sec (without seq num) 2024/11/23-14:41:14.201158 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264038: 929475 keys, 39133888 bytes, temperature: kWarm 2024/11/23-14:41:14.201247 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344074201193, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264038, "file_size": 39133888, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3934836997, "largest_seqno": 4033503346, "table_properties": {"data_size": 33151024, "index_size": 1756608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26025300, "raw_average_key_size": 28, "raw_value_size": 108192502, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929475, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344070, "oldest_key_time": 0, "file_creation_time": 1732344070, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264038, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:14.241499 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264039.sst) /dev/shm/ToplingTemp/Topling-78110-zK9A7v 2024/11/23-14:41:15.825838 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264039.sst): first pass time = 1.58's, 84.717'MB/sec 2024/11/23-14:41:17.567722 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264039.sst): old prealloc_size = 147639500, real_size = 37879312 2024/11/23-14:41:17.568313 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264039.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923723 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0359 GB avg-val = 117.30 avg-zval = 38.85 2024/11/23-14:41:17.569023 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264039.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:18.603178 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264039.sst): second pass time = 1.74's, 62.236'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 40.199'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3058.558'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.266'MB/sec write SST data time = 0.17's, 250.804'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 43706.075'MB/sec zip my value time = 1.74's, unzip length = 0.108'GB zip my value throughput = 62.236'MB/sec zip pipeline throughput = 62.236'MB/sec entries = 923723 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.30 avg-zval = 38.85 usrkeys = 923723 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.30 avg-zval = 38.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923723 bytes = 2886656 } seq expand size = 6466061 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0359 GB dict = 1.08 MB all = 0.0416 GB } UnZip/Zip{ index = 10.5141 value = 3.0190 dict = 1.00 all = 3.2246 } Zip/UnZip{ index = 0.0951 value = 0.3312 dict = 1.00 all = 0.3101 } ---------------------------- total value len = 9.078138 GB avg = 0.167 KB (by entry num) total key len = 1.492711 GB avg = 0.028 KB total ukey len = 1.034942 GB avg = 0.015 KB total ukey num = 0.067403633 Billion total entry num = 0.054211362 Billion write speed all = 0.509036459 MB/sec (with seq num) write speed all = 0.488152188 MB/sec (without seq num) 2024/11/23-14:41:18.654646 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264039: 923723 keys, 41623698 bytes, temperature: kWarm 2024/11/23-14:41:18.654704 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344078654672, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264039, "file_size": 41623698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777827, "largest_seqno": 4033503795, "table_properties": {"data_size": 35891072, "index_size": 1757120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25864244, "raw_average_key_size": 28, "raw_value_size": 108353582, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923723, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344074, "oldest_key_time": 0, "file_creation_time": 1732344074, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264039, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:18.691811 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264040.sst) /dev/shm/ToplingTemp/Topling-78110-o3suSu 2024/11/23-14:41:19.737566 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264040.sst): first pass time = 1.05's, 128.350'MB/sec 2024/11/23-14:41:22.208213 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264040.sst): old prealloc_size = 147639500, real_size = 36384832 2024/11/23-14:41:22.208794 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264040.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925844 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.97 avg-zval = 37.15 2024/11/23-14:41:22.209601 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264040.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:23.200077 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264040.sst): second pass time = 2.47's, 43.844'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 40.935'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3182.974'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.304'MB/sec write SST data time = 0.23's, 177.044'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36234.115'MB/sec zip my value time = 2.47's, unzip length = 0.108'GB zip my value throughput = 43.844'MB/sec zip pipeline throughput = 89.795'MB/sec entries = 925844 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 37.15 usrkeys = 925844 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 37.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925844 bytes = 3356192 } seq expand size = 6480908 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0406 GB } UnZip/Zip{ index = 10.5137 value = 3.1488 dict = 1.00 all = 3.3059 } Zip/UnZip{ index = 0.0951 value = 0.3176 dict = 1.00 all = 0.3025 } ---------------------------- total value len = 9.192913 GB avg = 0.167 KB (by entry num) total key len = 1.518634 GB avg = 0.028 KB total ukey len = 1.053459 GB avg = 0.015 KB total ukey num = 0.068329477 Billion total entry num = 0.055137206 Billion write speed all = 0.515697613 MB/sec (with seq num) write speed all = 0.494461372 MB/sec (without seq num) 2024/11/23-14:41:23.258325 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264040: 925844 keys, 40599730 bytes, temperature: kWarm 2024/11/23-14:41:23.258414 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344083258366, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264040, "file_size": 40599730, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4033501329, "table_properties": {"data_size": 34391984, "index_size": 1761216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923632, "raw_average_key_size": 28, "raw_value_size": 108294160, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925844, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344078, "oldest_key_time": 0, "file_creation_time": 1732344078, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264040, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:23.308959 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264041.sst) /dev/shm/ToplingTemp/Topling-78110-AEe2Wu 2024/11/23-14:41:24.449602 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264041.sst): first pass time = 1.14's, 117.675'MB/sec 2024/11/23-14:41:26.758099 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264041.sst): old prealloc_size = 147639500, real_size = 30092304 2024/11/23-14:41:26.758633 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264041.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931598 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0281 GB avg-val = 116.07 avg-zval = 30.14 2024/11/23-14:41:26.759375 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264041.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:27.943298 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264041.sst): second pass time = 2.31's, 46.865'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.19's, 28.760'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3479.566'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.245'MB/sec write SST data time = 0.24's, 144.504'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 40269.919'MB/sec zip my value time = 2.31's, unzip length = 0.108'GB zip my value throughput = 46.865'MB/sec zip pipeline throughput = 101.647'MB/sec entries = 931598 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.07 avg-zval = 30.14 usrkeys = 931598 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.07 avg-zval = 30.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931598 bytes = 3144160 } seq expand size = 6521186 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0281 GB dict = 1.08 MB all = 0.0341 GB } UnZip/Zip{ index = 10.4454 value = 3.8515 dict = 1.00 all = 3.9377 } Zip/UnZip{ index = 0.0957 value = 0.2596 dict = 1.00 all = 0.2540 } ---------------------------- total value len = 9.307568 GB avg = 0.166 KB (by entry num) total key len = 1.544719 GB avg = 0.028 KB total ukey len = 1.072091 GB avg = 0.015 KB total ukey num = 0.069261075 Billion total entry num = 0.056068804 Billion write speed all = 0.522354076 MB/sec (with seq num) write speed all = 0.500763958 MB/sec (without seq num) 2024/11/23-14:41:27.997913 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264041: 931598 keys, 34085570 bytes, temperature: kWarm 2024/11/23-14:41:27.998012 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344087997951, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264041, "file_size": 34085570, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3915567057, "largest_seqno": 4012776122, "table_properties": {"data_size": 28075456, "index_size": 1783744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26084744, "raw_average_key_size": 28, "raw_value_size": 108133012, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931598, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344083, "oldest_key_time": 0, "file_creation_time": 1732344083, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264041, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:28.069620 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264042.sst) /dev/shm/ToplingTemp/Topling-78110-0Wec9v 2024/11/23-14:41:29.006706 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264042.sst): first pass time = 0.94's, 143.235'MB/sec 2024/11/23-14:41:30.527463 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264042.sst): old prealloc_size = 147639500, real_size = 30311328 2024/11/23-14:41:30.528026 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264042.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931396 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0283 GB avg-val = 116.10 avg-zval = 30.37 2024/11/23-14:41:30.528832 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264042.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:31.593767 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264042.sst): second pass time = 1.52's, 71.167'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 32.501'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3304.487'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.268'MB/sec write SST data time = 0.20's, 174.750'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29768.659'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.167'MB/sec zip pipeline throughput = 81.415'MB/sec entries = 931396 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.10 avg-zval = 30.37 usrkeys = 931396 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.10 avg-zval = 30.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931396 bytes = 3492752 } seq expand size = 6519772 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.3798 value = 3.8234 dict = 1.00 all = 3.8730 } Zip/UnZip{ index = 0.0963 value = 0.2615 dict = 1.00 all = 0.2582 } ---------------------------- total value len = 9.422226 GB avg = 0.165 KB (by entry num) total key len = 1.570798 GB avg = 0.028 KB total ukey len = 1.090719 GB avg = 0.016 KB total ukey num = 0.070192471 Billion total entry num = 0.057000200 Billion write speed all = 0.529035256 MB/sec (with seq num) write speed all = 0.507090346 MB/sec (without seq num) 2024/11/23-14:41:31.636482 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264042: 931396 keys, 34655112 bytes, temperature: kWarm 2024/11/23-14:41:31.636542 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344091636509, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264042, "file_size": 34655112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776133, "table_properties": {"data_size": 28283664, "index_size": 1794624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26079088, "raw_average_key_size": 28, "raw_value_size": 108138768, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931396, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344088, "oldest_key_time": 0, "file_creation_time": 1732344088, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264042, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:31.678706 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264043.sst) /dev/shm/ToplingTemp/Topling-78110-PU2vDw 2024/11/23-14:41:32.596786 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264043.sst): first pass time = 0.92's, 146.200'MB/sec 2024/11/23-14:41:34.137427 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264043.sst): old prealloc_size = 147639500, real_size = 33116688 2024/11/23-14:41:34.138028 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264043.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928076 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.62 avg-zval = 33.52 2024/11/23-14:41:34.138853 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264043.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:35.174966 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264043.sst): second pass time = 1.54's, 70.310'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 36.096'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3053.563'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.13's, 292.294'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35162.982'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.310'MB/sec zip pipeline throughput = 70.310'MB/sec entries = 928076 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.62 avg-zval = 33.52 usrkeys = 928076 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.62 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928076 bytes = 3480304 } seq expand size = 6496532 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4449 value = 3.4793 dict = 1.00 all = 3.5839 } Zip/UnZip{ index = 0.0957 value = 0.2874 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 9.536954 GB avg = 0.165 KB (by entry num) total key len = 1.596784 GB avg = 0.028 KB total ukey len = 1.109280 GB avg = 0.016 KB total ukey num = 0.071120547 Billion total entry num = 0.057928276 Billion write speed all = 0.535714758 MB/sec (with seq num) write speed all = 0.513416384 MB/sec (without seq num) 2024/11/23-14:41:35.220421 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264043: 928076 keys, 37450588 bytes, temperature: kWarm 2024/11/23-14:41:35.220492 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344095220456, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264043, "file_size": 37450588, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3379308401, "largest_seqno": 4012776149, "table_properties": {"data_size": 31107392, "index_size": 1777088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986128, "raw_average_key_size": 28, "raw_value_size": 108231736, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928076, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344091, "oldest_key_time": 0, "file_creation_time": 1732344091, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264043, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:35.255966 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264044.sst) /dev/shm/ToplingTemp/Topling-78110-VNPRQt 2024/11/23-14:41:36.160414 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264044.sst): first pass time = 0.90's, 148.405'MB/sec 2024/11/23-14:41:37.639876 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264044.sst): old prealloc_size = 147639500, real_size = 32741088 2024/11/23-14:41:37.640501 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264044.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931972 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.01 avg-zval = 32.98 2024/11/23-14:41:37.641428 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264044.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:38.696136 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264044.sst): second pass time = 1.48's, 73.119'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 34.892'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2917.731'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.252'MB/sec write SST data time = 0.13's, 282.914'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33706.451'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.119'MB/sec zip pipeline throughput = 73.119'MB/sec entries = 931972 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.01 avg-zval = 32.98 usrkeys = 931972 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.01 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931972 bytes = 3261920 } seq expand size = 6523804 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5122 value = 3.5179 dict = 1.00 all = 3.6419 } Zip/UnZip{ index = 0.0951 value = 0.2843 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 9.651601 GB avg = 0.164 KB (by entry num) total key len = 1.622879 GB avg = 0.028 KB total ukey len = 1.127920 GB avg = 0.016 KB total ukey num = 0.072052519 Billion total entry num = 0.058860248 Billion write speed all = 0.542394830 MB/sec (with seq num) write speed all = 0.519741550 MB/sec (without seq num) 2024/11/23-14:41:38.742489 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264044: 931972 keys, 36853790 bytes, temperature: kWarm 2024/11/23-14:41:38.742562 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344098742525, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264044, "file_size": 36853790, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776176, "table_properties": {"data_size": 30734800, "index_size": 1773120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26095216, "raw_average_key_size": 28, "raw_value_size": 108122516, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931972, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344095, "oldest_key_time": 0, "file_creation_time": 1732344095, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264044, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:38.781407 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264045.sst) /dev/shm/ToplingTemp/Topling-78110-hFJUBx 2024/11/23-14:41:39.685839 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264045.sst): first pass time = 0.90's, 148.407'MB/sec 2024/11/23-14:41:41.188967 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264045.sst): old prealloc_size = 147639500, real_size = 32820208 2024/11/23-14:41:41.189505 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264045.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931885 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.03 avg-zval = 33.05 2024/11/23-14:41:41.190298 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264045.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:42.240114 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264045.sst): second pass time = 1.50's, 71.972'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.136'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3565.434'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.253'MB/sec write SST data time = 0.13's, 281.257'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25420.350'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 71.972'MB/sec zip pipeline throughput = 71.972'MB/sec entries = 931885 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.03 avg-zval = 33.05 usrkeys = 931885 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.03 avg-zval = 33.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931885 bytes = 3261616 } seq expand size = 6523195 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4479 value = 3.5102 dict = 1.00 all = 3.6341 } Zip/UnZip{ index = 0.0957 value = 0.2849 dict = 1.00 all = 0.2752 } ---------------------------- total value len = 9.766249 GB avg = 0.163 KB (by entry num) total key len = 1.648972 GB avg = 0.028 KB total ukey len = 1.146557 GB avg = 0.016 KB total ukey num = 0.072984404 Billion total entry num = 0.059792133 Billion write speed all = 0.549072013 MB/sec (with seq num) write speed all = 0.526064006 MB/sec (without seq num) 2024/11/23-14:41:42.285684 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264045: 931885 keys, 36932572 bytes, temperature: kWarm 2024/11/23-14:41:42.285756 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344102285717, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264045, "file_size": 36932572, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 4012776192, "table_properties": {"data_size": 30803168, "index_size": 1783872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26092780, "raw_average_key_size": 28, "raw_value_size": 108125074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931885, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344098, "oldest_key_time": 0, "file_creation_time": 1732344098, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264045, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:42.321882 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264046.sst) /dev/shm/ToplingTemp/Topling-78110-wmxE3u 2024/11/23-14:41:43.349301 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264046.sst): first pass time = 1.03's, 130.641'MB/sec 2024/11/23-14:41:44.729797 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264046.sst): old prealloc_size = 147639500, real_size = 33686800 2024/11/23-14:41:44.730455 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264046.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1036308 prefix = 4 raw-key = 0.0166 GB zip-key = 0.0020 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1052 GB zip-val = 0.0314 GB avg-val = 101.52 avg-zval = 30.34 2024/11/23-14:41:44.730904 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264046.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0026 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:45.787888 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264046.sst): second pass time = 1.38's, 76.247'MB/sec, value only(78.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 33.984'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3128.885'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.289'MB/sec write SST data time = 0.16's, 222.033'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.051'MB, throughput = 32990.806'MB/sec zip my value time = 1.38's, unzip length = 0.105'GB zip my value throughput = 76.247'MB/sec zip pipeline throughput = 76.247'MB/sec entries = 1036308 avg-key = 28.00 avg-zkey = 1.91 avg-val = 101.52 avg-zval = 30.34 usrkeys = 1036308 avg-key = 20.00 avg-zkey = 1.91 avg-val = 104.13 avg-zval = 30.34 TagRS{ kinds = RS_Key0_TagN bytes = 162040 } TagArray{ size = 386874 bytes = 1305712 } seq expand size = 2708118 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0207 GB value = 0.1052 GB dict = 1.05 MB all = 0.1342 GB } __ZipSize{ index = 0.0020 GB value = 0.0314 GB dict = 1.05 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4450 value = 3.3458 dict = 1.00 all = 3.7327 } Zip/UnZip{ index = 0.0957 value = 0.2989 dict = 1.00 all = 0.2679 } ---------------------------- total value len = 9.874158 GB avg = 0.162 KB (by entry num) total key len = 1.677989 GB avg = 0.028 KB total ukey len = 1.167284 GB avg = 0.016 KB total ukey num = 0.074020712 Billion total entry num = 0.060828441 Billion write speed all = 0.555563341 MB/sec (with seq num) write speed all = 0.532160556 MB/sec (without seq num) 2024/11/23-14:41:45.834295 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264046: 1036308 keys, 35957286 bytes, temperature: kWarm 2024/11/23-14:41:45.834367 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344105834330, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264046, "file_size": 35957286, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 31443200, "index_size": 1984320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 29016624, "raw_average_key_size": 28, "raw_value_size": 105201148, "raw_average_value_size": 101, "num_data_blocks": 1, "num_entries": 1036308, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344102, "oldest_key_time": 0, "file_creation_time": 1732344102, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264046, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:45.874009 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264047.sst) /dev/shm/ToplingTemp/Topling-78110-nuaIQt 2024/11/23-14:41:47.074545 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264047.sst): first pass time = 1.20's, 111.802'MB/sec 2024/11/23-14:41:48.587832 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264047.sst): old prealloc_size = 147639500, real_size = 36261328 2024/11/23-14:41:48.588567 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264047.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 977712 prefix = 4 raw-key = 0.0156 GB zip-key = 0.0019 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1068 GB zip-val = 0.0341 GB avg-val = 109.28 avg-zval = 34.92 2024/11/23-14:41:48.588950 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264047.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:49.445121 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264047.sst): second pass time = 1.51's, 70.644'MB/sec, value only(79.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 44.415'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2614.965'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.348'MB/sec write SST data time = 0.16's, 245.606'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.060'MB, throughput = 33603.620'MB/sec zip my value time = 1.51's, unzip length = 0.107'GB zip my value throughput = 70.644'MB/sec zip pipeline throughput = 70.644'MB/sec entries = 977712 avg-key = 28.00 avg-zkey = 1.91 avg-val = 109.28 avg-zval = 34.92 usrkeys = 977712 avg-key = 20.00 avg-zkey = 1.91 avg-val = 110.99 avg-zval = 34.92 TagRS{ kinds = RS_Key0_TagN bytes = 152880 } TagArray{ size = 238743 bytes = 835616 } seq expand size = 1671201 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0196 GB value = 0.1068 GB dict = 1.06 MB all = 0.1342 GB } __ZipSize{ index = 0.0019 GB value = 0.0341 GB dict = 1.06 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4460 value = 3.1291 dict = 1.00 all = 3.5251 } Zip/UnZip{ index = 0.0957 value = 0.3196 dict = 1.00 all = 0.2837 } ---------------------------- total value len = 9.982671 GB avg = 0.162 KB (by entry num) total key len = 1.705365 GB avg = 0.028 KB total ukey len = 1.186838 GB avg = 0.016 KB total ukey num = 0.074998424 Billion total entry num = 0.061806153 Billion write speed all = 0.561999638 MB/sec (with seq num) write speed all = 0.538224875 MB/sec (without seq num) 2024/11/23-14:41:49.495437 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264047: 977712 keys, 38075398 bytes, temperature: kWarm 2024/11/23-14:41:49.495504 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344109495468, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264047, "file_size": 38075398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3427302287, "table_properties": {"data_size": 34144768, "index_size": 1871936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 27375936, "raw_average_key_size": 28, "raw_value_size": 106841833, "raw_average_value_size": 109, "num_data_blocks": 1, "num_entries": 977712, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344105, "oldest_key_time": 0, "file_creation_time": 1732344105, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264047, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:49.532127 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264048.sst) /dev/shm/ToplingTemp/Topling-78110-zsE9Vu 2024/11/23-14:41:50.400815 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264048.sst): first pass time = 0.87's, 154.512'MB/sec 2024/11/23-14:41:51.684483 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264048.sst): old prealloc_size = 147639500, real_size = 31108320 2024/11/23-14:41:51.685076 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264048.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0291 GB avg-val = 116.10 avg-zval = 31.25 2024/11/23-14:41:51.685907 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264048.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:52.539513 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264048.sst): second pass time = 1.28's, 84.291'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.465'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3061.961'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 280.620'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35677.729'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.291'MB/sec zip pipeline throughput = 84.291'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 31.25 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 31.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3492880 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5145 value = 3.7156 dict = 1.00 all = 3.7859 } Zip/UnZip{ index = 0.0951 value = 0.2691 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 10.097329 GB avg = 0.161 KB (by entry num) total key len = 1.731445 GB avg = 0.028 KB total ukey len = 1.205466 GB avg = 0.016 KB total ukey num = 0.075929853 Billion total entry num = 0.062737582 Billion write speed all = 0.568682165 MB/sec (with seq num) write speed all = 0.544552703 MB/sec (without seq num) 2024/11/23-14:41:52.585067 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264048: 931429 keys, 35452448 bytes, temperature: kWarm 2024/11/23-14:41:52.585143 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344112585102, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264048, "file_size": 35452448, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3427302288, "largest_seqno": 4012775885, "table_properties": {"data_size": 29103568, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344109, "oldest_key_time": 0, "file_creation_time": 1732344109, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264048, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:52.624013 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264049.sst) /dev/shm/ToplingTemp/Topling-78110-Ecw5su 2024/11/23-14:41:53.503356 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264049.sst): first pass time = 0.88's, 152.641'MB/sec 2024/11/23-14:41:54.778626 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264049.sst): old prealloc_size = 147639500, real_size = 30877424 2024/11/23-14:41:54.779176 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264049.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0289 GB avg-val = 116.12 avg-zval = 31.00 2024/11/23-14:41:54.779935 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264049.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:55.609151 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264049.sst): second pass time = 1.27's, 84.855'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.129'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3361.934'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 275.388'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34859.286'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.855'MB/sec zip pipeline throughput = 84.855'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 31.00 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 31.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5123 value = 3.7455 dict = 1.00 all = 3.8361 } Zip/UnZip{ index = 0.0951 value = 0.2670 dict = 1.00 all = 0.2607 } ---------------------------- total value len = 10.211989 GB avg = 0.160 KB (by entry num) total key len = 1.757521 GB avg = 0.028 KB total ukey len = 1.224093 GB avg = 0.016 KB total ukey num = 0.076861162 Billion total entry num = 0.063668891 Billion write speed all = 0.575363350 MB/sec (with seq num) write speed all = 0.550879310 MB/sec (without seq num) 2024/11/23-14:41:55.654219 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264049: 931309 keys, 34988404 bytes, temperature: kWarm 2024/11/23-14:41:55.654287 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344115654252, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264049, "file_size": 34988404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3806919388, "largest_seqno": 4012775902, "table_properties": {"data_size": 28872544, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344112, "oldest_key_time": 0, "file_creation_time": 1732344112, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264049, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:55.690770 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264050.sst) /dev/shm/ToplingTemp/Topling-78110-uOfZzu 2024/11/23-14:41:56.702649 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264050.sst): first pass time = 1.01's, 132.647'MB/sec 2024/11/23-14:41:58.009340 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264050.sst): old prealloc_size = 147639500, real_size = 31145440 2024/11/23-14:41:58.009895 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264050.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0291 GB avg-val = 116.09 avg-zval = 31.27 2024/11/23-14:41:58.010724 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264050.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:41:58.832226 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264050.sst): second pass time = 1.31's, 82.803'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.130'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3343.952'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 281.293'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34895.333'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.803'MB/sec zip pipeline throughput = 82.803'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.09 avg-zval = 31.27 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.09 avg-zval = 31.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4435 value = 3.7124 dict = 1.00 all = 3.7819 } Zip/UnZip{ index = 0.0958 value = 0.2694 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 10.326646 GB avg = 0.160 KB (by entry num) total key len = 1.783602 GB avg = 0.028 KB total ukey len = 1.242722 GB avg = 0.016 KB total ukey num = 0.077792624 Billion total entry num = 0.064600353 Billion write speed all = 0.582038322 MB/sec (with seq num) write speed all = 0.557199934 MB/sec (without seq num) 2024/11/23-14:41:58.877989 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264050: 931462 keys, 35489604 bytes, temperature: kWarm 2024/11/23-14:41:58.878060 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344118878025, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264050, "file_size": 35489604, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 29128592, "index_size": 1783808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344115, "oldest_key_time": 0, "file_creation_time": 1732344115, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264050, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:41:58.917028 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264051.sst) /dev/shm/ToplingTemp/Topling-78110-ZAvHFv 2024/11/23-14:41:59.947220 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264051.sst): first pass time = 1.03's, 130.289'MB/sec 2024/11/23-14:42:01.239045 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264051.sst): old prealloc_size = 147639500, real_size = 32281280 2024/11/23-14:42:01.239581 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264051.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924886 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 117.12 avg-zval = 32.74 2024/11/23-14:42:01.241073 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264051.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:02.070765 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264051.sst): second pass time = 1.29's, 83.904'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.874'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3395.027'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 282.501'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34325.344'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.904'MB/sec zip pipeline throughput = 83.904'MB/sec entries = 924886 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.12 avg-zval = 32.74 usrkeys = 924886 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.12 avg-zval = 32.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924886 bytes = 3352720 } seq expand size = 6474202 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4455 value = 3.5774 dict = 1.00 all = 3.6782 } Zip/UnZip{ index = 0.0957 value = 0.2795 dict = 1.00 all = 0.2719 } ---------------------------- total value len = 10.441442 GB avg = 0.159 KB (by entry num) total key len = 1.809499 GB avg = 0.028 KB total ukey len = 1.261220 GB avg = 0.016 KB total ukey num = 0.078717510 Billion total entry num = 0.065525239 Billion write speed all = 0.588708578 MB/sec (with seq num) write speed all = 0.563518500 MB/sec (without seq num) 2024/11/23-14:42:02.116636 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264051: 924886 keys, 36489934 bytes, temperature: kWarm 2024/11/23-14:42:02.116709 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344122116673, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264051, "file_size": 36489934, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3611211802, "largest_seqno": 4012775947, "table_properties": {"data_size": 30279024, "index_size": 1770880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25896808, "raw_average_key_size": 28, "raw_value_size": 108321028, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924886, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344118, "oldest_key_time": 0, "file_creation_time": 1732344118, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264051, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:02.152514 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264052.sst) /dev/shm/ToplingTemp/Topling-78110-swd2ru 2024/11/23-14:42:03.043426 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264052.sst): first pass time = 0.89's, 150.660'MB/sec 2024/11/23-14:42:04.340167 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264052.sst): old prealloc_size = 147639500, real_size = 33973920 2024/11/23-14:42:04.340749 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264052.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923243 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.38 avg-zval = 34.63 2024/11/23-14:42:04.342249 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264052.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:05.197782 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264052.sst): second pass time = 1.30's, 83.621'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 44.519'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3200.678'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 295.036'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35087.810'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.621'MB/sec zip pipeline throughput = 83.621'MB/sec entries = 923243 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.38 avg-zval = 34.63 usrkeys = 923243 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.38 avg-zval = 34.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923243 bytes = 3346768 } seq expand size = 6462701 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4439 value = 3.3891 dict = 1.00 all = 3.5155 } Zip/UnZip{ index = 0.0957 value = 0.2951 dict = 1.00 all = 0.2845 } ---------------------------- total value len = 10.556271 GB avg = 0.159 KB (by entry num) total key len = 1.835350 GB avg = 0.028 KB total ukey len = 1.279684 GB avg = 0.016 KB total ukey num = 0.079640753 Billion total entry num = 0.066448482 Billion write speed all = 0.595379395 MB/sec (with seq num) write speed all = 0.569838229 MB/sec (without seq num) 2024/11/23-14:42:05.245554 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264052: 923243 keys, 38178640 bytes, temperature: kWarm 2024/11/23-14:42:05.245637 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344125245599, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264052, "file_size": 38178640, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 31974928, "index_size": 1768000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25850804, "raw_average_key_size": 28, "raw_value_size": 108366986, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923243, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344122, "oldest_key_time": 0, "file_creation_time": 1732344122, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264052, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:05.281256 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264053.sst) /dev/shm/ToplingTemp/Topling-78110-o6nmgu 2024/11/23-14:42:06.252285 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264053.sst): first pass time = 0.97's, 138.227'MB/sec 2024/11/23-14:42:07.559432 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264053.sst): old prealloc_size = 147639500, real_size = 33218048 2024/11/23-14:42:07.559963 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264053.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932147 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 115.99 avg-zval = 33.47 2024/11/23-14:42:07.561454 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264053.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:08.404408 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264053.sst): second pass time = 1.31's, 82.790'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.320'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3392.242'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 294.920'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34240.805'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.790'MB/sec zip pipeline throughput = 82.790'MB/sec entries = 932147 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.99 avg-zval = 33.47 usrkeys = 932147 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.99 avg-zval = 33.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932147 bytes = 3379040 } seq expand size = 6525029 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4448 value = 3.4653 dict = 1.00 all = 3.5841 } Zip/UnZip{ index = 0.0957 value = 0.2886 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 10.670914 GB avg = 0.158 KB (by entry num) total key len = 1.861450 GB avg = 0.028 KB total ukey len = 1.298327 GB avg = 0.016 KB total ukey num = 0.080572900 Billion total entry num = 0.067380629 Billion write speed all = 0.602048896 MB/sec (with seq num) write speed all = 0.576153425 MB/sec (without seq num) 2024/11/23-14:42:08.451166 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264053: 932147 keys, 37447978 bytes, temperature: kWarm 2024/11/23-14:42:08.451235 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344128451201, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264053, "file_size": 37447978, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012775987, "table_properties": {"data_size": 31199920, "index_size": 1784896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26100116, "raw_average_key_size": 28, "raw_value_size": 108117614, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932147, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344125, "oldest_key_time": 0, "file_creation_time": 1732344125, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264053, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:08.486881 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264054.sst) /dev/shm/ToplingTemp/Topling-78110-9Y5SWx 2024/11/23-14:42:09.520587 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264054.sst): first pass time = 1.03's, 129.846'MB/sec 2024/11/23-14:42:10.851751 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264054.sst): old prealloc_size = 147639500, real_size = 33594192 2024/11/23-14:42:10.852356 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264054.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929869 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0316 GB avg-val = 116.34 avg-zval = 33.96 2024/11/23-14:42:10.853161 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264054.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:11.697246 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264054.sst): second pass time = 1.33's, 81.339'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.679'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3029.737'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 291.437'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33758.806'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.339'MB/sec zip pipeline throughput = 81.339'MB/sec entries = 929869 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.34 avg-zval = 33.96 usrkeys = 929869 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.72 avg-zval = 33.96 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 847594 bytes = 3178496 } seq expand size = 5933158 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4459 value = 3.4255 dict = 1.00 all = 3.5531 } Zip/UnZip{ index = 0.0957 value = 0.2919 dict = 1.00 all = 0.2814 } ---------------------------- total value len = 10.785029 GB avg = 0.158 KB (by entry num) total key len = 1.887486 GB avg = 0.028 KB total ukey len = 1.316925 GB avg = 0.016 KB total ukey num = 0.081502769 Billion total entry num = 0.068310498 Billion write speed all = 0.608685396 MB/sec (with seq num) write speed all = 0.582436713 MB/sec (without seq num) 2024/11/23-14:42:11.744830 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264054: 929869 keys, 37774318 bytes, temperature: kWarm 2024/11/23-14:42:11.744897 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344131744861, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264054, "file_size": 37774318, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 31581184, "index_size": 1780352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26036332, "raw_average_key_size": 28, "raw_value_size": 108181454, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929869, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344128, "oldest_key_time": 0, "file_creation_time": 1732344128, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264054, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:11.781437 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264055.sst) /dev/shm/ToplingTemp/Topling-78110-Z79rgv 2024/11/23-14:42:12.700236 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264055.sst): first pass time = 0.92's, 146.086'MB/sec 2024/11/23-14:42:14.008003 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264055.sst): old prealloc_size = 147639500, real_size = 32567136 2024/11/23-14:42:14.008509 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264055.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929780 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.35 avg-zval = 32.89 2024/11/23-14:42:14.009296 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264055.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:14.835429 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264055.sst): second pass time = 1.31's, 82.784'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.327'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3645.665'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 283.145'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34403.061'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.784'MB/sec zip pipeline throughput = 82.784'MB/sec entries = 929780 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.35 avg-zval = 32.89 usrkeys = 929780 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.60 avg-zval = 32.89 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 829058 bytes = 3108976 } seq expand size = 5803406 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5838 value = 3.5380 dict = 1.00 all = 3.6594 } Zip/UnZip{ index = 0.0945 value = 0.2826 dict = 1.00 all = 0.2733 } ---------------------------- total value len = 10.899016 GB avg = 0.157 KB (by entry num) total key len = 1.913520 GB avg = 0.028 KB total ukey len = 1.335520 GB avg = 0.016 KB total ukey num = 0.082432549 Billion total entry num = 0.069240278 Billion write speed all = 0.615318134 MB/sec (with seq num) write speed all = 0.588716188 MB/sec (without seq num) 2024/11/23-14:42:14.881264 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264055: 929780 keys, 36677424 bytes, temperature: kWarm 2024/11/23-14:42:14.881332 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344134881297, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264055, "file_size": 36677424, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 30577488, "index_size": 1756992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26033840, "raw_average_key_size": 28, "raw_value_size": 108183912, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929780, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344131, "oldest_key_time": 0, "file_creation_time": 1732344131, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264055, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:14.919960 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264056.sst) /dev/shm/ToplingTemp/Topling-78110-Rqvz3x 2024/11/23-14:42:15.929162 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264056.sst): first pass time = 1.01's, 133.002'MB/sec 2024/11/23-14:42:17.222421 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264056.sst): old prealloc_size = 147639500, real_size = 31781952 2024/11/23-14:42:17.222962 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264056.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931692 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0298 GB avg-val = 116.06 avg-zval = 31.96 2024/11/23-14:42:17.223769 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264056.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:18.064812 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264056.sst): second pass time = 1.29's, 83.683'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.749'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3292.003'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 283.359'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34549.090'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.683'MB/sec zip pipeline throughput = 83.683'MB/sec entries = 931692 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 31.96 usrkeys = 931692 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 31.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931692 bytes = 3377392 } seq expand size = 6521844 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5136 value = 3.6314 dict = 1.00 all = 3.7272 } Zip/UnZip{ index = 0.0951 value = 0.2754 dict = 1.00 all = 0.2683 } ---------------------------- total value len = 11.013668 GB avg = 0.157 KB (by entry num) total key len = 1.939608 GB avg = 0.028 KB total ukey len = 1.354154 GB avg = 0.016 KB total ukey num = 0.083364241 Billion total entry num = 0.070171970 Billion write speed all = 0.621980649 MB/sec (with seq num) write speed all = 0.595024930 MB/sec (without seq num) 2024/11/23-14:42:18.109346 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264056: 931692 keys, 36010448 bytes, temperature: kWarm 2024/11/23-14:42:18.109418 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344138109382, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264056, "file_size": 36010448, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776228, "table_properties": {"data_size": 29776496, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26087376, "raw_average_key_size": 28, "raw_value_size": 108130408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931692, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344134, "oldest_key_time": 0, "file_creation_time": 1732344134, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264056, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:18.147552 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264057.sst) /dev/shm/ToplingTemp/Topling-78110-bxXOdw 2024/11/23-14:42:19.035789 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264057.sst): first pass time = 0.89's, 151.112'MB/sec 2024/11/23-14:42:20.348064 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264057.sst): old prealloc_size = 147639500, real_size = 33174144 2024/11/23-14:42:20.348603 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264057.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933210 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 115.82 avg-zval = 33.38 2024/11/23-14:42:20.349382 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264057.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:21.198821 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264057.sst): second pass time = 1.31's, 82.415'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.971'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3449.905'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 297.734'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34313.230'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.415'MB/sec zip pipeline throughput = 82.415'MB/sec entries = 933210 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.82 avg-zval = 33.38 usrkeys = 933210 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.82 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933210 bytes = 3382896 } seq expand size = 6532470 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4448 value = 3.4695 dict = 1.00 all = 3.5880 } Zip/UnZip{ index = 0.0957 value = 0.2882 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 11.128288 GB avg = 0.157 KB (by entry num) total key len = 1.965738 GB avg = 0.028 KB total ukey len = 1.372818 GB avg = 0.016 KB total ukey num = 0.084297451 Billion total entry num = 0.071105180 Billion write speed all = 0.628644483 MB/sec (with seq num) write speed all = 0.601334393 MB/sec (without seq num) 2024/11/23-14:42:21.245067 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264057: 933210 keys, 37407708 bytes, temperature: kWarm 2024/11/23-14:42:21.245134 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344141245099, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264057, "file_size": 37407708, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 4012776268, "table_properties": {"data_size": 31153712, "index_size": 1786944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26129880, "raw_average_key_size": 28, "raw_value_size": 108087848, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933210, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344138, "oldest_key_time": 0, "file_creation_time": 1732344138, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264057, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:21.283623 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264058.sst) /dev/shm/ToplingTemp/Topling-78110-ShGGQu 2024/11/23-14:42:22.187430 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264058.sst): first pass time = 0.90's, 148.508'MB/sec 2024/11/23-14:42:23.490291 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264058.sst): old prealloc_size = 147639500, real_size = 32726256 2024/11/23-14:42:23.490811 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264058.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924396 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.20 avg-zval = 33.24 2024/11/23-14:42:23.492884 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264058.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:24.326882 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264058.sst): second pass time = 1.30's, 83.238'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.287'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3528.084'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 290.497'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 36512.793'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.238'MB/sec zip pipeline throughput = 83.238'MB/sec entries = 924396 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.20 avg-zval = 33.24 usrkeys = 924396 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.20 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924396 bytes = 3466496 } seq expand size = 6470772 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4430 value = 3.5260 dict = 1.00 all = 3.6227 } Zip/UnZip{ index = 0.0958 value = 0.2836 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 11.243094 GB avg = 0.156 KB (by entry num) total key len = 1.991621 GB avg = 0.028 KB total ukey len = 1.391306 GB avg = 0.016 KB total ukey num = 0.085221847 Billion total entry num = 0.072029576 Billion write speed all = 0.635303535 MB/sec (with seq num) write speed all = 0.607642557 MB/sec (without seq num) 2024/11/23-14:42:24.372548 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264058: 924396 keys, 37049168 bytes, temperature: kWarm 2024/11/23-14:42:24.372626 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344144372581, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264058, "file_size": 37049168, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 30724576, "index_size": 1770368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25883088, "raw_average_key_size": 28, "raw_value_size": 108334704, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924396, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344141, "oldest_key_time": 0, "file_creation_time": 1732344141, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264058, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:24.413047 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264059.sst) /dev/shm/ToplingTemp/Topling-78110-5QZYfu 2024/11/23-14:42:25.463846 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264059.sst): first pass time = 1.05's, 127.734'MB/sec 2024/11/23-14:42:26.825705 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264059.sst): old prealloc_size = 147639500, real_size = 34417440 2024/11/23-14:42:26.826265 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264059.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929627 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.38 avg-zval = 34.86 2024/11/23-14:42:26.828347 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264059.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:27.673051 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264059.sst): second pass time = 1.36's, 79.488'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.602'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3317.053'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 302.450'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27432.639'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.488'MB/sec zip pipeline throughput = 79.488'MB/sec entries = 929627 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.38 avg-zval = 34.86 usrkeys = 929627 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.38 avg-zval = 34.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929627 bytes = 3369920 } seq expand size = 6507389 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4458 value = 3.3386 dict = 1.00 all = 3.4736 } Zip/UnZip{ index = 0.0957 value = 0.2995 dict = 1.00 all = 0.2879 } ---------------------------- total value len = 11.357790 GB avg = 0.156 KB (by entry num) total key len = 2.017650 GB avg = 0.028 KB total ukey len = 1.409899 GB avg = 0.016 KB total ukey num = 0.086151474 Billion total entry num = 0.072959203 Billion write speed all = 0.641955626 MB/sec (with seq num) write speed all = 0.613942149 MB/sec (without seq num) 2024/11/23-14:42:27.719447 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264059: 929627 keys, 38639728 bytes, temperature: kWarm 2024/11/23-14:42:27.719516 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344147719481, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264059, "file_size": 38639728, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 32404944, "index_size": 1779904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26029556, "raw_average_key_size": 28, "raw_value_size": 108188238, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929627, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344144, "oldest_key_time": 0, "file_creation_time": 1732344144, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264059, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:27.755140 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264060.sst) /dev/shm/ToplingTemp/Topling-78110-bI7bpv 2024/11/23-14:42:28.750071 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264060.sst): first pass time = 0.99's, 134.907'MB/sec 2024/11/23-14:42:30.045521 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264060.sst): old prealloc_size = 147639500, real_size = 31370112 2024/11/23-14:42:30.046101 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264060.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929427 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0294 GB avg-val = 116.41 avg-zval = 31.59 2024/11/23-14:42:30.048247 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264060.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:30.880240 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264060.sst): second pass time = 1.29's, 83.569'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.780'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3188.552'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 276.041'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35560.301'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.569'MB/sec zip pipeline throughput = 83.569'MB/sec entries = 929427 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.41 avg-zval = 31.59 usrkeys = 929427 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.41 avg-zval = 31.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929427 bytes = 3485360 } seq expand size = 6505989 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4436 value = 3.6854 dict = 1.00 all = 3.7588 } Zip/UnZip{ index = 0.0958 value = 0.2713 dict = 1.00 all = 0.2660 } ---------------------------- total value len = 11.472489 GB avg = 0.155 KB (by entry num) total key len = 2.043674 GB avg = 0.028 KB total ukey len = 1.428487 GB avg = 0.016 KB total ukey num = 0.087080901 Billion total entry num = 0.073888630 Billion write speed all = 0.648609842 MB/sec (with seq num) write speed all = 0.620243868 MB/sec (without seq num) 2024/11/23-14:42:30.924492 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264060: 929427 keys, 35708000 bytes, temperature: kWarm 2024/11/23-14:42:30.924559 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344150924524, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264060, "file_size": 35708000, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 29357680, "index_size": 1779904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023956, "raw_average_key_size": 28, "raw_value_size": 108193910, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344147, "oldest_key_time": 0, "file_creation_time": 1732344147, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264060, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:30.969583 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264061.sst) /dev/shm/ToplingTemp/Topling-78110-s6oWTx 2024/11/23-14:42:31.781639 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264061.sst): first pass time = 0.81's, 165.290'MB/sec 2024/11/23-14:42:33.060909 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264061.sst): old prealloc_size = 147639500, real_size = 32297312 2024/11/23-14:42:33.061461 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264061.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923122 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.40 avg-zval = 32.81 2024/11/23-14:42:33.063576 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264061.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:33.887833 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264061.sst): second pass time = 1.28's, 84.762'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.283'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.799'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 287.687'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36159.169'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.762'MB/sec zip pipeline throughput = 84.762'MB/sec entries = 923122 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.40 avg-zval = 32.81 usrkeys = 923122 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.40 avg-zval = 32.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923122 bytes = 3461728 } seq expand size = 6461854 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.3753 value = 3.5781 dict = 1.00 all = 3.6655 } Zip/UnZip{ index = 0.0964 value = 0.2795 dict = 1.00 all = 0.2728 } ---------------------------- total value len = 11.587322 GB avg = 0.155 KB (by entry num) total key len = 2.069522 GB avg = 0.028 KB total ukey len = 1.446950 GB avg = 0.016 KB total ukey num = 0.088004023 Billion total entry num = 0.074811752 Billion write speed all = 0.655266167 MB/sec (with seq num) write speed all = 0.626549950 MB/sec (without seq num) 2024/11/23-14:42:33.933678 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264061: 923122 keys, 36616874 bytes, temperature: kWarm 2024/11/23-14:42:33.933747 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344153933713, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264061, "file_size": 36616874, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776360, "table_properties": {"data_size": 30286864, "index_size": 1779456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847416, "raw_average_key_size": 28, "raw_value_size": 108370436, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923122, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344150, "oldest_key_time": 0, "file_creation_time": 1732344150, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264061, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:33.969842 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264062.sst) /dev/shm/ToplingTemp/Topling-78110-pjbN7u 2024/11/23-14:42:34.949491 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264062.sst): first pass time = 0.98's, 137.011'MB/sec 2024/11/23-14:42:36.428011 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264062.sst): old prealloc_size = 147639500, real_size = 36278864 2024/11/23-14:42:36.428543 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264062.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930066 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0343 GB avg-val = 116.31 avg-zval = 36.84 2024/11/23-14:42:36.428786 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264062.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:37.145911 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264062.sst): second pass time = 1.48's, 73.200'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 52.626'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3487.307'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.392'MB/sec write SST data time = 0.12's, 304.908'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32576.692'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.200'MB/sec zip pipeline throughput = 73.200'MB/sec entries = 930066 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.31 avg-zval = 36.84 usrkeys = 930066 avg-key = 20.00 avg-zkey = 1.92 avg-val = 117.51 avg-zval = 36.84 TagRS{ kinds = RS_Key0_TagN bytes = 145440 } TagArray{ size = 159664 bytes = 498960 } seq expand size = 1117648 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4432 value = 3.1570 dict = 1.00 all = 3.5526 } Zip/UnZip{ index = 0.0958 value = 0.3168 dict = 1.00 all = 0.2815 } ---------------------------- total value len = 11.696615 GB avg = 0.154 KB (by entry num) total key len = 2.095563 GB avg = 0.028 KB total ukey len = 1.465551 GB avg = 0.016 KB total ukey num = 0.088934089 Billion total entry num = 0.075741818 Billion write speed all = 0.661656239 MB/sec (with seq num) write speed all = 0.632587563 MB/sec (without seq num) 2024/11/23-14:42:37.194345 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264062: 930066 keys, 37779911 bytes, temperature: kWarm 2024/11/23-14:42:37.194412 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344157194378, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264062, "file_size": 37779911, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012773626, "table_properties": {"data_size": 34264960, "index_size": 1781184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26041848, "raw_average_key_size": 28, "raw_value_size": 108176020, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930066, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344153, "oldest_key_time": 0, "file_creation_time": 1732344153, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264062, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:37.229038 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264063.sst) /dev/shm/ToplingTemp/Topling-78110-I1kAyx 2024/11/23-14:42:38.282237 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264063.sst): first pass time = 1.05's, 127.443'MB/sec 2024/11/23-14:42:39.732404 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264063.sst): old prealloc_size = 147639500, real_size = 37551728 2024/11/23-14:42:39.733000 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264063.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925461 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0355 GB avg-val = 117.03 avg-zval = 38.41 2024/11/23-14:42:39.734013 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264063.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:40.470860 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264063.sst): second pass time = 1.45's, 74.761'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 54.215'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3082.844'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.378'MB/sec write SST data time = 0.13's, 317.522'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35525.228'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.761'MB/sec zip pipeline throughput = 74.761'MB/sec entries = 925461 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.03 avg-zval = 38.41 usrkeys = 925461 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.01 avg-zval = 38.41 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 394270 bytes = 1478528 } seq expand size = 2759890 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0355 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4463 value = 3.0467 dict = 1.00 all = 3.3526 } Zip/UnZip{ index = 0.0957 value = 0.3282 dict = 1.00 all = 0.2983 } ---------------------------- total value len = 11.807680 GB avg = 0.154 KB (by entry num) total key len = 2.121476 GB avg = 0.028 KB total ukey len = 1.484060 GB avg = 0.017 KB total ukey num = 0.089859550 Billion total entry num = 0.076667279 Billion write speed all = 0.668120939 MB/sec (with seq num) write speed all = 0.638701777 MB/sec (without seq num) 2024/11/23-14:42:40.520232 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264063: 925461 keys, 40033823 bytes, temperature: kWarm 2024/11/23-14:42:40.520301 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344160520266, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264063, "file_size": 40033823, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4033503187, "table_properties": {"data_size": 35548320, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912908, "raw_average_key_size": 28, "raw_value_size": 108304914, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925461, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344157, "oldest_key_time": 0, "file_creation_time": 1732344157, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264063, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:40.557346 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264064.sst) /dev/shm/ToplingTemp/Topling-78110-fwjAOw 2024/11/23-14:42:41.643879 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264064.sst): first pass time = 1.09's, 123.533'MB/sec 2024/11/23-14:42:42.990285 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264064.sst): old prealloc_size = 147639500, real_size = 38155472 2024/11/23-14:42:42.990819 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264064.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923561 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0362 GB avg-val = 117.33 avg-zval = 39.16 2024/11/23-14:42:42.991615 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264064.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:42:43.785860 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264064.sst): second pass time = 1.35's, 80.522'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 52.667'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3419.313'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.345'MB/sec write SST data time = 0.13's, 332.387'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33981.420'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.522'MB/sec zip pipeline throughput = 80.522'MB/sec entries = 923561 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 39.16 usrkeys = 923561 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 39.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923561 bytes = 2886144 } seq expand size = 6464927 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0362 GB dict = 1.08 MB all = 0.0419 GB } UnZip/Zip{ index = 10.5122 value = 2.9960 dict = 1.00 all = 3.2033 } Zip/UnZip{ index = 0.0951 value = 0.3338 dict = 1.00 all = 0.3122 } ---------------------------- total value len = 11.922503 GB avg = 0.154 KB (by entry num) total key len = 2.147336 GB avg = 0.028 KB total ukey len = 1.502532 GB avg = 0.017 KB total ukey num = 0.090783111 Billion total entry num = 0.077590840 Billion write speed all = 0.674761588 MB/sec (with seq num) write speed all = 0.644992766 MB/sec (without seq num) 2024/11/23-14:42:43.836260 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264064: 923561 keys, 41899308 bytes, temperature: kWarm 2024/11/23-14:42:43.836328 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344163836292, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264064, "file_size": 41899308, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012776767, "largest_seqno": 4033503446, "table_properties": {"data_size": 36167296, "index_size": 1757120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859708, "raw_average_key_size": 28, "raw_value_size": 108358154, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923561, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344160, "oldest_key_time": 0, "file_creation_time": 1732344160, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264064, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:43.877686 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264065.sst) /dev/shm/ToplingTemp/Topling-78110-4sCbsx 2024/11/23-14:42:43.947270 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264065.sst): first pass time = 0.07's, 235.135'MB/sec 2024/11/23-14:42:44.084856 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264065.sst): old prealloc_size = 147639500, real_size = 5089536 2024/11/23-14:42:44.084971 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264065.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 112578 prefix = 4 raw-key = 0.0018 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0132 GB zip-val = 0.0048 GB avg-val = 117.26 avg-zval = 43.03 2024/11/23-14:42:44.085136 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264065.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.016 GB 2024/11/23-14:42:44.179459 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00/264065.sst): second pass time = 0.14's, 96.386'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.09's, 59.450'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1803.919'MB/sec (index lex order gen) rebuild zvType time = 0.08's, 0.359'MB/sec write SST data time = 0.02's, 349.952'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.138'MB, throughput = 4748.334'MB/sec zip my value time = 0.14's, unzip length = 0.013'GB zip my value throughput = 96.386'MB/sec zip pipeline throughput = 96.386'MB/sec entries = 112578 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.26 avg-zval = 43.03 usrkeys = 112578 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.26 avg-zval = 43.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 112578 bytes = 422176 } seq expand size = 788046 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0023 GB value = 0.0132 GB dict = 0.14 MB all = 0.0164 GB } __ZipSize{ index = 0.0002 GB value = 0.0048 GB dict = 0.14 MB all = 0.0056 GB } UnZip/Zip{ index = 10.3808 value = 2.7249 dict = 1.00 all = 2.9080 } Zip/UnZip{ index = 0.0963 value = 0.3670 dict = 1.00 all = 0.3439 } ---------------------------- total value len = 11.936492 GB avg = 0.154 KB (by entry num) total key len = 2.150488 GB avg = 0.028 KB total ukey len = 1.504783 GB avg = 0.017 KB total ukey num = 0.090895689 Billion total entry num = 0.077703418 Billion write speed all = 0.675570871 MB/sec (with seq num) write speed all = 0.645759419 MB/sec (without seq num) 2024/11/23-14:42:44.194522 139957500515904 [compaction_job.cc:1948] [default] [JOB 28164] Generated table #264065: 112578 keys, 5623301 bytes, temperature: kWarm 2024/11/23-14:42:44.194576 139957500515904 EVENT_LOG_v1 {"time_micros": 1732344164194547, "cf_name": "default", "job": 28164, "event": "table_file_creation", "file_number": 264065, "file_size": 5623301, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3236084454, "largest_seqno": 4031990312, "table_properties": {"data_size": 4844336, "index_size": 216896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3152184, "raw_average_key_size": 28, "raw_value_size": 13200492, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 112578, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732344163, "oldest_key_time": 0, "file_creation_time": 1732344163, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 264065, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:42:44.215919 139957500515904 [compaction_job.cc:703] job-28164: subcompact[0], size: 1.014635 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28164/att-00 [264038,264039,264040,264041,264042,264043,264044,264045,264046,264047,264048,264049,264050,264051,264052,264053,264054,264055,264056,264057,264058,264059,264060,264061,264062,264063,264064,264065] 2024/11/23-14:42:44.377561 139957500515904 (Original Log Time 2024/11/23-14:41:09.748126) EVENT_LOG_v1 {"time_micros": 1732344069748076, "job": 28164, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [264034], "files_L3": [263950, 263983, 263984, 263985, 263986, 263987, 263988, 263989, 263990, 263991, 263992, 263993, 263994, 263995, 263996, 263997, 263998, 263999, 264000, 264001, 264002, 264003, 264004, 264005, 264006, 264007, 264008], "score": 0.998396, "input_data_size": 1040568744, "oldest_snapshot_seqno": 4033532489} 2024/11/23-14:42:44.377565 139957500515904 (Original Log Time 2024/11/23-14:42:44.375992) [compaction_job.cc:2022] [default] [JOB 28164] Compacted 1@2 + 27@3 files to L3 => 1014634651 bytes 2024/11/23-14:42:44.377566 139957500515904 (Original Log Time 2024/11/23-14:42:44.377478) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 28 0 0 0] max score 0.04, MB/sec: 11.0 rd, 10.7 wr, level 3, files in(1, 27) out(28 +0 blob) MB in(24.8, 967.6 +0.0 blob) out(967.6 +0.0 blob), read-write-amplify(79.2) write-amplify(39.1) OK, records in: 25445643, records dropped: 100389 output_compression: Snappy 2024/11/23-14:42:44.377570 139957500515904 (Original Log Time 2024/11/23-14:42:44.377528) EVENT_LOG_v1 {"time_micros": 1732344164377501, "cf": "default", "job": 28164, "event": "compaction_finished", "compaction_time_micros": 94467799, "compaction_time_cpu_micros": 40559802, "output_level": 3, "num_output_files": 28, "total_output_size": 1014634651, "num_input_records": 25445643, "num_output_records": 25345254, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 738538866, "file_range_sync_nanos": 0, "file_fsync_nanos": 20302, "file_prepare_write_nanos": 75111, "lsm_state": [0, 0, 0, 28, 0, 0, 0]} 2024/11/23-14:42:44.510027 139957500515904 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732344069,"executesMs":94775,"compactionJobId":28164,"attempt":0,"compactionInputRawBytes":3654488571,"compactionInputZipBytes":1040568744,"compactionOutputRawBytes":0,"compactionOutputZipBytes":1014634651,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-9b107aace5eac7ecc1efa0365f1d737c-019a6a97a77f0311-00"}}, response = 2024/11/23-14:42:44.510113 139957500515904 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28164/att-00: olev 3, work 94.775 s, result 7.759 ms, install 1.591 ms, input{raw 3.404 GiB zip 992.364 MiB}