2024/11/23-18:15:14.441243 139957576050240 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00: fork to child time = 0.011252 sec 2024/11/23-18:15:14.441987 139957576050240 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00 2024/11/23-18:15:14.450662 139957576050240 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00 2024/11/23-18:15:14.451930 139957576050240 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-18:15:14.452287 139957576050240 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-18:15:14.504341 139957576050240 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32861/att-00/MANIFEST-206347 2024/11/23-18:15:14.520277 139957576050240 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32861/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 274297, last_sequence is 4175758501, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 274134 2024/11/23-18:15:14.520286 139957576050240 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-18:15:14.520458 139957576050240 [version_set.cc:5721] Creating manifest 274297 2024/11/23-18:15:14.556466 139957576050240 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-18:15:14.581252 139957576050240 [compaction_job.cc:2437] [default] [JOB 32861] Compacting 1@2 + 20@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-18:15:14.581277 139957576050240 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24598 Base level 2, inputs: [274264(27MB)], [274286(39MB) 274287(3949KB) 274221(37MB) 274222(35MB) 274223(36MB) 274224(38MB) 274225(37MB) 274226(35MB) 274227(35MB) 274228(36MB) 274229(33MB) 274230(37MB) 274231(33MB) 274232(33MB) 274233(35MB) 274234(37MB) 274235(36MB) 274236(37MB) 274237(36MB) 274288(34MB)] 2024/11/23-18:15:14.590384 139957576050240 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:15:14.590461 139957576050240 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:15:14.590507 139957576050240 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:15:14.590521 139957576050240 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:15:14.590551 139957576050240 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:15:15.224228 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274298.sst) /dev/shm/ToplingTemp/Topling-146296-2QlDGp 2024/11/23-18:15:15.937305 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274298.sst): first pass time = 0.71's, 188.237'MB/sec 2024/11/23-18:15:17.886755 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274298.sst): old prealloc_size = 147639500, real_size = 36802784 2024/11/23-18:15:17.887383 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274298.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930202 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0348 GB avg-val = 116.29 avg-zval = 37.41 2024/11/23-18:15:17.888303 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274298.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:18.936565 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274298.sst): second pass time = 1.95's, 55.512'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 39.195'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2936.882'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.22's, 187.291'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 21893.444'MB/sec zip my value time = 1.95's, unzip length = 0.108'GB zip my value throughput = 55.512'MB/sec zip pipeline throughput = 119.441'MB/sec entries = 930202 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.29 avg-zval = 37.41 usrkeys = 930202 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.29 avg-zval = 37.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930202 bytes = 3488272 } seq expand size = 6511414 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.5135 value = 3.1083 dict = 1.00 all = 3.2619 } Zip/UnZip{ index = 0.0951 value = 0.3217 dict = 1.00 all = 0.3066 } ---------------------------- total value len = 8.963303 GB avg = 0.168 KB (by entry num) total key len = 1.466867 GB avg = 0.028 KB total ukey len = 1.016482 GB avg = 0.015 KB total ukey num = 0.066480637 Billion total entry num = 0.053288366 Billion write speed all = 0.310359638 MB/sec (with seq num) write speed all = 0.297674470 MB/sec (without seq num) 2024/11/23-18:15:18.986901 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274298: 930202 keys, 41146802 bytes, temperature: kWarm 2024/11/23-18:15:18.987015 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356918986936, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274298, "file_size": 41146802, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4175509265, "table_properties": {"data_size": 34800528, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26045656, "raw_average_key_size": 28, "raw_value_size": 108172108, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930202, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356915, "oldest_key_time": 0, "file_creation_time": 1732356915, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274298, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:19.023504 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274299.sst) /dev/shm/ToplingTemp/Topling-146296-vdbEdo 2024/11/23-18:15:20.025716 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274299.sst): first pass time = 1.00's, 133.926'MB/sec 2024/11/23-18:15:21.602490 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274299.sst): old prealloc_size = 147639500, real_size = 36490704 2024/11/23-18:15:21.603090 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274299.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934275 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0345 GB avg-val = 115.66 avg-zval = 36.88 2024/11/23-18:15:21.603734 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274299.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:22.557898 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274299.sst): second pass time = 1.58's, 68.566'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 41.705'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3115.171'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.283'MB/sec write SST data time = 0.13's, 309.366'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34860.922'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.566'MB/sec zip pipeline throughput = 68.566'MB/sec entries = 934275 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.66 avg-zval = 36.88 usrkeys = 934275 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.36 avg-zval = 36.88 TagRS{ kinds = RS_Key0_TagN bytes = 146080 } TagArray{ size = 627483 bytes = 2353072 } seq expand size = 4392381 multi value expand size = 0 cnt WriteAppend = 20 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.3764 value = 3.1361 dict = 1.00 all = 3.3686 } Zip/UnZip{ index = 0.0964 value = 0.3189 dict = 1.00 all = 0.2969 } ---------------------------- total value len = 9.075754 GB avg = 0.167 KB (by entry num) total key len = 1.493026 GB avg = 0.028 KB total ukey len = 1.035168 GB avg = 0.015 KB total ukey num = 0.067414912 Billion total entry num = 0.054222641 Billion write speed all = 0.314450230 MB/sec (with seq num) write speed all = 0.301544051 MB/sec (without seq num) 2024/11/23-18:15:22.611034 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274299: 934275 keys, 39843978 bytes, temperature: kWarm 2024/11/23-18:15:22.611105 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356922611068, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274299, "file_size": 39843978, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34456192, "index_size": 1800768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26159700, "raw_average_key_size": 28, "raw_value_size": 108058038, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934275, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356919, "oldest_key_time": 0, "file_creation_time": 1732356919, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274299, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:22.647657 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274300.sst) /dev/shm/ToplingTemp/Topling-146296-bgjtVr 2024/11/23-18:15:23.522011 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274300.sst): first pass time = 0.87's, 153.511'MB/sec 2024/11/23-18:15:25.065430 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274300.sst): old prealloc_size = 147639500, real_size = 31897936 2024/11/23-18:15:25.066030 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274300.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0299 GB avg-val = 116.09 avg-zval = 32.09 2024/11/23-18:15:25.066847 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274300.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:26.084280 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274300.sst): second pass time = 1.54's, 70.103'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 35.572'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3069.163'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.262'MB/sec write SST data time = 0.13's, 282.101'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33963.966'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.103'MB/sec zip pipeline throughput = 70.103'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 32.09 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5118 value = 3.6175 dict = 1.00 all = 3.7034 } Zip/UnZip{ index = 0.0951 value = 0.2764 dict = 1.00 all = 0.2700 } ---------------------------- total value len = 9.190411 GB avg = 0.167 KB (by entry num) total key len = 1.519107 GB avg = 0.028 KB total ukey len = 1.053797 GB avg = 0.015 KB total ukey num = 0.068346374 Billion total entry num = 0.055154103 Billion write speed all = 0.318604145 MB/sec (with seq num) write speed all = 0.305477634 MB/sec (without seq num) 2024/11/23-18:15:26.133150 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274300: 931462 keys, 36242208 bytes, temperature: kWarm 2024/11/23-18:15:26.133217 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356926133181, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274300, "file_size": 36242208, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 29892672, "index_size": 1772224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356922, "oldest_key_time": 0, "file_creation_time": 1732356922, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274300, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:26.169621 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274301.sst) /dev/shm/ToplingTemp/Topling-146296-llP2Cp 2024/11/23-18:15:27.052497 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274301.sst): first pass time = 0.88's, 152.030'MB/sec 2024/11/23-18:15:28.682556 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274301.sst): old prealloc_size = 147639500, real_size = 33086336 2024/11/23-18:15:28.683141 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274301.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.04 avg-zval = 33.59 2024/11/23-18:15:28.683931 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274301.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:29.708627 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274301.sst): second pass time = 1.63's, 66.481'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 36.349'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3155.767'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.13's, 291.238'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34953.971'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.481'MB/sec zip pipeline throughput = 66.481'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.04 avg-zval = 33.59 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.04 avg-zval = 33.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4457 value = 3.4844 dict = 1.00 all = 3.5987 } Zip/UnZip{ index = 0.0957 value = 0.2870 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 9.305195 GB avg = 0.166 KB (by entry num) total key len = 1.545019 GB avg = 0.028 KB total ukey len = 1.072305 GB avg = 0.015 KB total ukey num = 0.069271779 Billion total entry num = 0.056079508 Billion write speed all = 0.322754986 MB/sec (with seq num) write speed all = 0.309409670 MB/sec (without seq num) 2024/11/23-18:15:29.757667 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274301: 925405 keys, 37296106 bytes, temperature: kWarm 2024/11/23-18:15:29.757738 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356929757701, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274301, "file_size": 37296106, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 31082992, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356926, "oldest_key_time": 0, "file_creation_time": 1732356926, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274301, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:29.795541 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274302.sst) /dev/shm/ToplingTemp/Topling-146296-AQdKco 2024/11/23-18:15:30.958987 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274302.sst): first pass time = 1.16's, 115.367'MB/sec 2024/11/23-18:15:32.468506 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274302.sst): old prealloc_size = 147639500, real_size = 35859152 2024/11/23-18:15:32.469215 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274302.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924151 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0339 GB avg-val = 117.23 avg-zval = 36.64 2024/11/23-18:15:32.470127 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274302.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:33.501922 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274302.sst): second pass time = 1.51's, 71.869'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 38.771'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2587.169'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.275'MB/sec write SST data time = 0.19's, 207.833'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30038.703'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.869'MB/sec zip pipeline throughput = 71.869'MB/sec entries = 924151 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.23 avg-zval = 36.64 usrkeys = 924151 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.23 avg-zval = 36.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924151 bytes = 3350064 } seq expand size = 6469057 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.4436 value = 3.1999 dict = 1.00 all = 3.3499 } Zip/UnZip{ index = 0.0958 value = 0.3125 dict = 1.00 all = 0.2985 } ---------------------------- total value len = 9.420006 GB avg = 0.165 KB (by entry num) total key len = 1.570895 GB avg = 0.028 KB total ukey len = 1.090788 GB avg = 0.016 KB total ukey num = 0.070195930 Billion total entry num = 0.057003659 Billion write speed all = 0.326903028 MB/sec (with seq num) write speed all = 0.313339322 MB/sec (without seq num) 2024/11/23-18:15:33.556347 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274302: 924151 keys, 40065822 bytes, temperature: kWarm 2024/11/23-18:15:33.556419 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356933556380, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274302, "file_size": 40065822, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33858112, "index_size": 1769792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876228, "raw_average_key_size": 28, "raw_value_size": 108341550, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924151, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356929, "oldest_key_time": 0, "file_creation_time": 1732356929, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274302, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:33.602042 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274303.sst) /dev/shm/ToplingTemp/Topling-146296-2uOgpp 2024/11/23-18:15:34.826973 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274303.sst): first pass time = 1.22's, 109.575'MB/sec 2024/11/23-18:15:36.402524 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274303.sst): old prealloc_size = 147639500, real_size = 34167984 2024/11/23-18:15:36.403110 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274303.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930410 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0322 GB avg-val = 116.26 avg-zval = 34.56 2024/11/23-18:15:36.403872 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274303.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:37.245034 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274303.sst): second pass time = 1.56's, 69.153'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.221'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3161.184'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 294.525'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 28069.358'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.153'MB/sec zip pipeline throughput = 69.153'MB/sec entries = 930410 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.26 avg-zval = 34.56 usrkeys = 930410 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.13 avg-zval = 34.56 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 780691 bytes = 2927600 } seq expand size = 5464837 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4467 value = 3.3640 dict = 1.00 all = 3.5230 } Zip/UnZip{ index = 0.0957 value = 0.2973 dict = 1.00 all = 0.2838 } ---------------------------- total value len = 9.533637 GB avg = 0.165 KB (by entry num) total key len = 1.596946 GB avg = 0.028 KB total ukey len = 1.109396 GB avg = 0.016 KB total ukey num = 0.071126340 Billion total entry num = 0.057934069 Billion write speed all = 0.331020766 MB/sec (with seq num) write speed all = 0.317237208 MB/sec (without seq num) 2024/11/23-18:15:37.296964 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274303: 930410 keys, 38097186 bytes, temperature: kWarm 2024/11/23-18:15:37.297058 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356937297018, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274303, "file_size": 38097186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 32153952, "index_size": 1781248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26051480, "raw_average_key_size": 28, "raw_value_size": 108166372, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930410, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356933, "oldest_key_time": 0, "file_creation_time": 1732356933, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274303, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:37.334486 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274304.sst) /dev/shm/ToplingTemp/Topling-146296-Byb1bq 2024/11/23-18:15:38.341278 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274304.sst): first pass time = 1.01's, 133.317'MB/sec 2024/11/23-18:15:39.668112 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274304.sst): old prealloc_size = 147639500, real_size = 32652048 2024/11/23-18:15:39.668662 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274304.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.44 avg-zval = 33.00 2024/11/23-18:15:39.669535 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274304.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:40.538114 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274304.sst): second pass time = 1.33's, 81.594'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.546'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3322.169'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.14's, 268.414'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32540.159'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.594'MB/sec zip pipeline throughput = 81.594'MB/sec entries = 929240 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.44 avg-zval = 33.00 usrkeys = 929240 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.19 avg-zval = 33.00 TagRS{ kinds = RS_Key0_TagN bytes = 145280 } TagArray{ size = 895962 bytes = 3359872 } seq expand size = 6271734 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5834 value = 3.5286 dict = 1.00 all = 3.6262 } Zip/UnZip{ index = 0.0945 value = 0.2834 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 9.648108 GB avg = 0.164 KB (by entry num) total key len = 1.622965 GB avg = 0.028 KB total ukey len = 1.127981 GB avg = 0.016 KB total ukey num = 0.072055580 Billion total entry num = 0.058863309 Billion write speed all = 0.335166066 MB/sec (with seq num) write speed all = 0.321162797 MB/sec (without seq num) 2024/11/23-18:15:40.585691 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274304: 929240 keys, 37013772 bytes, temperature: kWarm 2024/11/23-18:15:40.585766 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356940585729, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274304, "file_size": 37013772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 30663552, "index_size": 1756032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26018720, "raw_average_key_size": 28, "raw_value_size": 108199112, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356937, "oldest_key_time": 0, "file_creation_time": 1732356937, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274304, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:40.625551 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274305.sst) /dev/shm/ToplingTemp/Topling-146296-TkOCTo 2024/11/23-18:15:41.687353 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274305.sst): first pass time = 1.06's, 126.411'MB/sec 2024/11/23-18:15:43.432012 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274305.sst): old prealloc_size = 147639500, real_size = 32153680 2024/11/23-18:15:43.432571 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274305.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931692 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0301 GB avg-val = 116.06 avg-zval = 32.36 2024/11/23-18:15:43.434215 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274305.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:44.284446 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274305.sst): second pass time = 1.74's, 62.007'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.681'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3338.813'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 281.216'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35211.262'MB/sec zip my value time = 1.74's, unzip length = 0.108'GB zip my value throughput = 62.007'MB/sec zip pipeline throughput = 120.462'MB/sec entries = 931692 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 32.36 usrkeys = 931692 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 32.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931692 bytes = 3377392 } seq expand size = 6521844 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5136 value = 3.5866 dict = 1.00 all = 3.6891 } Zip/UnZip{ index = 0.0951 value = 0.2788 dict = 1.00 all = 0.2711 } ---------------------------- total value len = 9.762760 GB avg = 0.163 KB (by entry num) total key len = 1.649053 GB avg = 0.028 KB total ukey len = 1.146615 GB avg = 0.016 KB total ukey num = 0.072987272 Billion total entry num = 0.059795001 Billion write speed all = 0.339313417 MB/sec (with seq num) write speed all = 0.325090087 MB/sec (without seq num) 2024/11/23-18:15:44.330231 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274305: 931692 keys, 36381980 bytes, temperature: kWarm 2024/11/23-18:15:44.330305 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356944330266, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274305, "file_size": 36381980, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776228, "table_properties": {"data_size": 30148224, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26087376, "raw_average_key_size": 28, "raw_value_size": 108130408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931692, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356940, "oldest_key_time": 0, "file_creation_time": 1732356940, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274305, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:44.366356 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274306.sst) /dev/shm/ToplingTemp/Topling-146296-O8az9r 2024/11/23-18:15:45.405347 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274306.sst): first pass time = 1.04's, 129.185'MB/sec 2024/11/23-18:15:46.766172 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274306.sst): old prealloc_size = 147639500, real_size = 33691824 2024/11/23-18:15:46.766748 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274306.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933210 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0317 GB avg-val = 115.82 avg-zval = 33.94 2024/11/23-18:15:46.767570 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274306.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:47.617226 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274306.sst): second pass time = 1.36's, 79.479'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.563'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3226.939'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 292.273'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27773.231'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.479'MB/sec zip pipeline throughput = 79.479'MB/sec entries = 933210 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.82 avg-zval = 33.94 usrkeys = 933210 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.82 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933210 bytes = 3382896 } seq expand size = 6532470 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4444 value = 3.4128 dict = 1.00 all = 3.5390 } Zip/UnZip{ index = 0.0957 value = 0.2930 dict = 1.00 all = 0.2826 } ---------------------------- total value len = 9.877380 GB avg = 0.163 KB (by entry num) total key len = 1.675182 GB avg = 0.028 KB total ukey len = 1.165279 GB avg = 0.016 KB total ukey num = 0.073920482 Billion total entry num = 0.060728211 Billion write speed all = 0.343464380 MB/sec (with seq num) write speed all = 0.329020501 MB/sec (without seq num) 2024/11/23-18:15:47.664926 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274306: 933210 keys, 37925016 bytes, temperature: kWarm 2024/11/23-18:15:47.665008 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356947664962, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274306, "file_size": 37925016, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 4012776268, "table_properties": {"data_size": 31671328, "index_size": 1787008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26129880, "raw_average_key_size": 28, "raw_value_size": 108087848, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933210, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356944, "oldest_key_time": 0, "file_creation_time": 1732356944, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274306, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:47.706531 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274307.sst) /dev/shm/ToplingTemp/Topling-146296-Dxtwmq 2024/11/23-18:15:48.696295 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274307.sst): first pass time = 0.99's, 135.611'MB/sec 2024/11/23-18:15:49.991252 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274307.sst): old prealloc_size = 147639500, real_size = 32045712 2024/11/23-18:15:49.991800 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274307.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926281 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0300 GB avg-val = 116.90 avg-zval = 32.43 2024/11/23-18:15:49.993471 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274307.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:50.821548 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274307.sst): second pass time = 1.29's, 83.667'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.810'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3417.119'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 285.608'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34860.416'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.667'MB/sec zip pipeline throughput = 83.667'MB/sec entries = 926281 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.90 avg-zval = 32.43 usrkeys = 926281 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.90 avg-zval = 32.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926281 bytes = 3473568 } seq expand size = 6483967 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4435 value = 3.6046 dict = 1.00 all = 3.6899 } Zip/UnZip{ index = 0.0958 value = 0.2774 dict = 1.00 all = 0.2710 } ---------------------------- total value len = 9.992146 GB avg = 0.162 KB (by entry num) total key len = 1.701118 GB avg = 0.028 KB total ukey len = 1.183805 GB avg = 0.016 KB total ukey num = 0.074846763 Billion total entry num = 0.061654492 Billion write speed all = 0.347614411 MB/sec (with seq num) write speed all = 0.332951618 MB/sec (without seq num) 2024/11/23-18:15:50.866067 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274307: 926281 keys, 36373978 bytes, temperature: kWarm 2024/11/23-18:15:50.866141 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356950866104, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274307, "file_size": 36373978, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 30040064, "index_size": 1773888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935868, "raw_average_key_size": 28, "raw_value_size": 108281978, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926281, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356947, "oldest_key_time": 0, "file_creation_time": 1732356947, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274307, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:50.905830 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274308.sst) /dev/shm/ToplingTemp/Topling-146296-SQgFas 2024/11/23-18:15:51.962550 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274308.sst): first pass time = 1.06's, 127.019'MB/sec 2024/11/23-18:15:53.311364 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274308.sst): old prealloc_size = 147639500, real_size = 34613616 2024/11/23-18:15:53.311894 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274308.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928430 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0326 GB avg-val = 116.56 avg-zval = 35.13 2024/11/23-18:15:53.313293 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274308.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:54.180213 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274308.sst): second pass time = 1.35's, 80.319'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 44.695'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3487.527'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 299.618'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33983.690'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.319'MB/sec zip pipeline throughput = 80.319'MB/sec entries = 928430 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 35.13 usrkeys = 928430 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 35.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928430 bytes = 3365568 } seq expand size = 6499010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5140 value = 3.3181 dict = 1.00 all = 3.4563 } Zip/UnZip{ index = 0.0951 value = 0.3014 dict = 1.00 all = 0.2893 } ---------------------------- total value len = 10.106867 GB avg = 0.161 KB (by entry num) total key len = 1.727114 GB avg = 0.028 KB total ukey len = 1.202373 GB avg = 0.016 KB total ukey num = 0.075775193 Billion total entry num = 0.062582922 Billion write speed all = 0.351762484 MB/sec (with seq num) write speed all = 0.336880376 MB/sec (without seq num) 2024/11/23-18:15:54.228241 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274308: 928430 keys, 38832298 bytes, temperature: kWarm 2024/11/23-18:15:54.228313 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356954228277, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274308, "file_size": 38832298, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 32615264, "index_size": 1766080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996040, "raw_average_key_size": 28, "raw_value_size": 108221808, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356950, "oldest_key_time": 0, "file_creation_time": 1732356950, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274308, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:54.266354 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274309.sst) /dev/shm/ToplingTemp/Topling-146296-7E32kq 2024/11/23-18:15:55.211550 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274309.sst): first pass time = 0.95's, 142.006'MB/sec 2024/11/23-18:15:56.530808 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274309.sst): old prealloc_size = 147639500, real_size = 30270640 2024/11/23-18:15:56.531421 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274309.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930624 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0283 GB avg-val = 116.22 avg-zval = 30.36 2024/11/23-18:15:56.532253 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274309.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:15:57.347543 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274309.sst): second pass time = 1.32's, 82.037'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.380'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3018.475'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 273.560'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29918.204'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.037'MB/sec zip pipeline throughput = 82.037'MB/sec entries = 930624 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.22 avg-zval = 30.36 usrkeys = 930624 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.22 avg-zval = 30.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930624 bytes = 3489856 } seq expand size = 6514368 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4439 value = 3.8279 dict = 1.00 all = 3.8778 } Zip/UnZip{ index = 0.0957 value = 0.2612 dict = 1.00 all = 0.2579 } ---------------------------- total value len = 10.221542 GB avg = 0.161 KB (by entry num) total key len = 1.753172 GB avg = 0.028 KB total ukey len = 1.220986 GB avg = 0.016 KB total ukey num = 0.076705817 Billion total entry num = 0.063513546 Billion write speed all = 0.355912209 MB/sec (with seq num) write speed all = 0.340810221 MB/sec (without seq num) 2024/11/23-18:15:57.391349 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274309: 930624 keys, 34611930 bytes, temperature: kWarm 2024/11/23-18:15:57.391422 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356957391384, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274309, "file_size": 34611930, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 28255648, "index_size": 1782144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26057472, "raw_average_key_size": 28, "raw_value_size": 108160340, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930624, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356954, "oldest_key_time": 0, "file_creation_time": 1732356954, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274309, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:15:57.429313 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274310.sst) /dev/shm/ToplingTemp/Topling-146296-IdgmTr 2024/11/23-18:15:58.372088 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274310.sst): first pass time = 0.94's, 142.372'MB/sec 2024/11/23-18:15:59.659522 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274310.sst): old prealloc_size = 147639500, real_size = 31506608 2024/11/23-18:15:59.660105 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274310.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923122 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0295 GB avg-val = 117.40 avg-zval = 31.95 2024/11/23-18:15:59.660921 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274310.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:16:00.477457 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274310.sst): second pass time = 1.29's, 84.224'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.802'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3165.475'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 280.964'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 26981.660'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.224'MB/sec zip pipeline throughput = 84.224'MB/sec entries = 923122 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.40 avg-zval = 31.95 usrkeys = 923122 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.40 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923122 bytes = 3461728 } seq expand size = 6461854 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.3753 value = 3.6741 dict = 1.00 all = 3.7464 } Zip/UnZip{ index = 0.0964 value = 0.2722 dict = 1.00 all = 0.2669 } ---------------------------- total value len = 10.336374 GB avg = 0.160 KB (by entry num) total key len = 1.779019 GB avg = 0.028 KB total ukey len = 1.239448 GB avg = 0.016 KB total ukey num = 0.077628939 Billion total entry num = 0.064436668 Billion write speed all = 0.360059993 MB/sec (with seq num) write speed all = 0.344739935 MB/sec (without seq num) 2024/11/23-18:16:00.520330 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274310: 923122 keys, 35826086 bytes, temperature: kWarm 2024/11/23-18:16:00.520410 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356960520372, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274310, "file_size": 35826086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776360, "table_properties": {"data_size": 29496160, "index_size": 1779456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847416, "raw_average_key_size": 28, "raw_value_size": 108370436, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923122, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356957, "oldest_key_time": 0, "file_creation_time": 1732356957, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274310, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:16:00.556184 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274311.sst) /dev/shm/ToplingTemp/Topling-146296-Bl11Pq 2024/11/23-18:16:01.531344 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274311.sst): first pass time = 0.98's, 137.642'MB/sec 2024/11/23-18:16:03.022289 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274311.sst): old prealloc_size = 147639500, real_size = 36581824 2024/11/23-18:16:03.022837 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274311.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928766 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0346 GB avg-val = 116.51 avg-zval = 37.22 2024/11/23-18:16:03.023047 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274311.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:16:03.730489 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274311.sst): second pass time = 1.49's, 72.621'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.457'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3412.760'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.399'MB/sec write SST data time = 0.13's, 302.189'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32998.074'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.621'MB/sec zip pipeline throughput = 72.621'MB/sec entries = 928766 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.51 avg-zval = 37.22 usrkeys = 928766 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.20 avg-zval = 37.22 TagRS{ kinds = RS_Key0_TagN bytes = 145200 } TagArray{ size = 90875 bytes = 272640 } seq expand size = 636125 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4440 value = 3.1302 dict = 1.00 all = 3.5454 } Zip/UnZip{ index = 0.0957 value = 0.3195 dict = 1.00 all = 0.2821 } ---------------------------- total value len = 10.445223 GB avg = 0.160 KB (by entry num) total key len = 1.805025 GB avg = 0.028 KB total ukey len = 1.258023 GB avg = 0.016 KB total ukey num = 0.078557705 Billion total entry num = 0.065365434 Billion write speed all = 0.364032553 MB/sec (with seq num) write speed all = 0.348493179 MB/sec (without seq num) 2024/11/23-18:16:03.777995 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274311: 928766 keys, 37857399 bytes, temperature: kWarm 2024/11/23-18:16:03.778072 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356963778033, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274311, "file_size": 37857399, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012773626, "table_properties": {"data_size": 34570864, "index_size": 1778560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005448, "raw_average_key_size": 28, "raw_value_size": 108212384, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928766, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356960, "oldest_key_time": 0, "file_creation_time": 1732356960, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274311, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:16:03.824547 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274312.sst) /dev/shm/ToplingTemp/Topling-146296-CJZkcs 2024/11/23-18:16:04.666833 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274312.sst): first pass time = 0.84's, 159.357'MB/sec 2024/11/23-18:16:06.118782 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274312.sst): old prealloc_size = 147639500, real_size = 36743648 2024/11/23-18:16:06.119346 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274312.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925730 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0347 GB avg-val = 116.99 avg-zval = 37.53 2024/11/23-18:16:06.119852 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274312.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:16:06.887680 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274312.sst): second pass time = 1.45's, 74.662'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 51.348'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3283.748'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.362'MB/sec write SST data time = 0.13's, 306.185'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35229.275'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.662'MB/sec zip pipeline throughput = 74.662'MB/sec entries = 925730 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.99 avg-zval = 37.53 usrkeys = 925730 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.48 avg-zval = 37.53 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 462028 bytes = 1732624 } seq expand size = 3234196 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4460 value = 3.1174 dict = 1.00 all = 3.3997 } Zip/UnZip{ index = 0.0957 value = 0.3208 dict = 1.00 all = 0.2941 } ---------------------------- total value len = 10.556754 GB avg = 0.159 KB (by entry num) total key len = 1.830945 GB avg = 0.028 KB total ukey len = 1.276538 GB avg = 0.016 KB total ukey num = 0.079483435 Billion total entry num = 0.066291164 Billion write speed all = 0.368082591 MB/sec (with seq num) write speed all = 0.352324621 MB/sec (without seq num) 2024/11/23-18:16:06.936519 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274312: 925730 keys, 39479841 bytes, temperature: kWarm 2024/11/23-18:16:06.936591 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356966936555, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274312, "file_size": 39479841, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4175509365, "table_properties": {"data_size": 34739600, "index_size": 1772416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920440, "raw_average_key_size": 28, "raw_value_size": 108297380, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925730, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356963, "oldest_key_time": 0, "file_creation_time": 1732356963, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274312, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:16:06.973696 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274313.sst) /dev/shm/ToplingTemp/Topling-146296-OwMCip 2024/11/23-18:16:08.014843 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274313.sst): first pass time = 1.04's, 128.919'MB/sec 2024/11/23-18:16:09.371409 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274313.sst): old prealloc_size = 147639500, real_size = 35190624 2024/11/23-18:16:09.371968 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274313.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926830 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 116.81 avg-zval = 35.82 2024/11/23-18:16:09.372786 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274313.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:16:10.207521 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274313.sst): second pass time = 1.36's, 79.883'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.853'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3004.758'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 308.131'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33236.199'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.883'MB/sec zip pipeline throughput = 79.883'MB/sec entries = 926830 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 35.82 usrkeys = 926830 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 35.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926830 bytes = 3128064 } seq expand size = 6487810 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5138 value = 3.2615 dict = 1.00 all = 3.4263 } Zip/UnZip{ index = 0.0951 value = 0.3066 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 10.671509 GB avg = 0.159 KB (by entry num) total key len = 1.856896 GB avg = 0.028 KB total ukey len = 1.295075 GB avg = 0.016 KB total ukey num = 0.080410265 Billion total entry num = 0.067217994 Billion write speed all = 0.372226738 MB/sec (with seq num) write speed all = 0.356250030 MB/sec (without seq num) 2024/11/23-18:16:10.256089 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274313: 926830 keys, 39172926 bytes, temperature: kWarm 2024/11/23-18:16:10.256165 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356970256127, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274313, "file_size": 39172926, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4175509401, "table_properties": {"data_size": 33195664, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951240, "raw_average_key_size": 28, "raw_value_size": 108266620, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926830, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356966, "oldest_key_time": 0, "file_creation_time": 1732356966, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274313, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:16:10.290760 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274314.sst) /dev/shm/ToplingTemp/Topling-146296-um9sFr 2024/11/23-18:16:11.353119 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274314.sst): first pass time = 1.06's, 126.344'MB/sec 2024/11/23-18:16:12.700874 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274314.sst): old prealloc_size = 147639500, real_size = 35558224 2024/11/23-18:16:12.701470 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274314.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929237 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0336 GB avg-val = 116.44 avg-zval = 36.11 2024/11/23-18:16:12.702291 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274314.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:16:13.529071 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274314.sst): second pass time = 1.35's, 80.327'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.612'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3067.519'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 311.921'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33989.041'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.327'MB/sec zip pipeline throughput = 80.327'MB/sec entries = 929237 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.44 avg-zval = 36.11 usrkeys = 929237 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.44 avg-zval = 36.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929237 bytes = 3020032 } seq expand size = 6504659 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.5129 value = 3.2242 dict = 1.00 all = 3.4039 } Zip/UnZip{ index = 0.0951 value = 0.3102 dict = 1.00 all = 0.2938 } ---------------------------- total value len = 10.786212 GB avg = 0.158 KB (by entry num) total key len = 1.882915 GB avg = 0.028 KB total ukey len = 1.313659 GB avg = 0.016 KB total ukey num = 0.081339502 Billion total entry num = 0.068147231 Billion write speed all = 0.376370546 MB/sec (with seq num) write speed all = 0.360174570 MB/sec (without seq num) 2024/11/23-18:16:13.577217 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274314: 929237 keys, 39431112 bytes, temperature: kWarm 2024/11/23-18:16:13.577288 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356973577251, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274314, "file_size": 39431112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4116739076, "largest_seqno": 4175509468, "table_properties": {"data_size": 33557952, "index_size": 1767808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26018636, "raw_average_key_size": 28, "raw_value_size": 108199098, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929237, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356970, "oldest_key_time": 0, "file_creation_time": 1732356970, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274314, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:16:13.619107 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274315.sst) /dev/shm/ToplingTemp/Topling-146296-Ho6XNp 2024/11/23-18:16:14.557813 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274315.sst): first pass time = 0.94's, 142.988'MB/sec 2024/11/23-18:16:15.856769 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274315.sst): old prealloc_size = 147639500, real_size = 33580960 2024/11/23-18:16:15.857326 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274315.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934102 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0316 GB avg-val = 115.69 avg-zval = 33.80 2024/11/23-18:16:15.858121 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274315.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:16:16.704539 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274315.sst): second pass time = 1.30's, 83.272'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.339'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3252.456'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 294.430'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34216.547'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.272'MB/sec zip pipeline throughput = 83.272'MB/sec entries = 934102 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.69 avg-zval = 33.80 usrkeys = 934102 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.69 avg-zval = 33.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934102 bytes = 3152608 } seq expand size = 6538714 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5146 value = 3.4229 dict = 1.00 all = 3.5707 } Zip/UnZip{ index = 0.0951 value = 0.2921 dict = 1.00 all = 0.2801 } ---------------------------- total value len = 10.900814 GB avg = 0.158 KB (by entry num) total key len = 1.909070 GB avg = 0.028 KB total ukey len = 1.332341 GB avg = 0.016 KB total ukey num = 0.082273604 Billion total entry num = 0.069081333 Billion write speed all = 0.380516203 MB/sec (with seq num) write speed all = 0.364099776 MB/sec (without seq num) 2024/11/23-18:16:16.751943 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274315: 934102 keys, 37588158 bytes, temperature: kWarm 2024/11/23-18:16:16.752040 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356976751998, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274315, "file_size": 37588158, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4076718630, "largest_seqno": 4175509505, "table_properties": {"data_size": 31570512, "index_size": 1776768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26154856, "raw_average_key_size": 28, "raw_value_size": 108063012, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934102, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356973, "oldest_key_time": 0, "file_creation_time": 1732356973, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274315, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:16:16.788507 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274316.sst) /dev/shm/ToplingTemp/Topling-146296-Ubo8Yq 2024/11/23-18:16:17.810819 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274316.sst): first pass time = 1.02's, 131.293'MB/sec 2024/11/23-18:16:19.089682 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274316.sst): old prealloc_size = 147639500, real_size = 30443216 2024/11/23-18:16:19.090250 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274316.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927958 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.64 avg-zval = 30.64 2024/11/23-18:16:19.091027 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274316.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:16:19.915079 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274316.sst): second pass time = 1.28's, 84.683'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.713'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.642'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 268.284'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32841.990'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.683'MB/sec zip pipeline throughput = 84.683'MB/sec entries = 927958 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.64 avg-zval = 30.64 usrkeys = 927958 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.64 avg-zval = 30.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927958 bytes = 3131872 } seq expand size = 6495706 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.4458 value = 3.8065 dict = 1.00 all = 3.8984 } Zip/UnZip{ index = 0.0957 value = 0.2627 dict = 1.00 all = 0.2565 } ---------------------------- total value len = 11.015545 GB avg = 0.157 KB (by entry num) total key len = 1.935053 GB avg = 0.028 KB total ukey len = 1.350901 GB avg = 0.016 KB total ukey num = 0.083201562 Billion total entry num = 0.070009291 Billion write speed all = 0.384659400 MB/sec (with seq num) write speed all = 0.368024040 MB/sec (without seq num) 2024/11/23-18:16:19.956613 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274316: 927958 keys, 34428856 bytes, temperature: kWarm 2024/11/23-18:16:19.956686 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356979956650, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274316, "file_size": 34428856, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4077094389, "largest_seqno": 4175508086, "table_properties": {"data_size": 28434368, "index_size": 1776704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982824, "raw_average_key_size": 28, "raw_value_size": 108235044, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927958, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356976, "oldest_key_time": 0, "file_creation_time": 1732356976, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274316, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:16:20.000433 139957576050240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274317.sst) /dev/shm/ToplingTemp/Topling-146296-b7EHes 2024/11/23-18:16:20.147672 139957576050240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274317.sst): first pass time = 0.15's, 243.425'MB/sec 2024/11/23-18:16:20.499850 139957576050240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274317.sst): old prealloc_size = 147639500, real_size = 9966704 2024/11/23-18:16:20.500074 139957576050240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274317.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 245425 prefix = 4 raw-key = 0.0039 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0290 GB zip-val = 0.0094 GB avg-val = 118.00 avg-zval = 38.44 2024/11/23-18:16:20.500299 139957576050240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274317.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.036 GB 2024/11/23-18:16:20.722591 139957576050240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00/274317.sst): second pass time = 0.35's, 82.405'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 49.258'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2333.517'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.327'MB/sec write SST data time = 0.03's, 314.779'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.297'MB, throughput = 9973.843'MB/sec zip my value time = 0.35's, unzip length = 0.029'GB zip my value throughput = 82.405'MB/sec zip pipeline throughput = 82.405'MB/sec entries = 245425 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 38.44 usrkeys = 245425 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 38.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 245425 bytes = 766976 } seq expand size = 1717975 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0049 GB value = 0.0290 GB dict = 0.30 MB all = 0.0358 GB } __ZipSize{ index = 0.0005 GB value = 0.0094 GB dict = 0.30 MB all = 0.0110 GB } UnZip/Zip{ index = 10.4220 value = 3.0697 dict = 1.00 all = 3.2661 } Zip/UnZip{ index = 0.0960 value = 0.3258 dict = 1.00 all = 0.3062 } ---------------------------- total value len = 11.046223 GB avg = 0.157 KB (by entry num) total key len = 1.941925 GB avg = 0.028 KB total ukey len = 1.355809 GB avg = 0.016 KB total ukey num = 0.083446987 Billion total entry num = 0.070254716 Billion write speed all = 0.385765461 MB/sec (with seq num) write speed all = 0.369072184 MB/sec (without seq num) 2024/11/23-18:16:20.741208 139957576050240 [compaction_job.cc:1948] [default] [JOB 32861] Generated table #274317: 245425 keys, 10970987 bytes, temperature: kWarm 2024/11/23-18:16:20.741266 139957576050240 EVENT_LOG_v1 {"time_micros": 1732356980741236, "cf_name": "default", "job": 32861, "event": "table_file_creation", "file_number": 274317, "file_size": 10970987, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4124279718, "largest_seqno": 4156487361, "table_properties": {"data_size": 9434208, "index_size": 470976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6871900, "raw_average_key_size": 28, "raw_value_size": 28960150, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 245425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356979, "oldest_key_time": 0, "file_creation_time": 1732356979, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274317, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:16:20.754056 139957576050240 [compaction_job.cc:703] job-32861: subcompact[0], size: 0.728586 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32861/att-00 [274298,274299,274300,274301,274302,274303,274304,274305,274306,274307,274308,274309,274310,274311,274312,274313,274314,274315,274316,274317] 2024/11/23-18:16:20.873305 139957576050240 (Original Log Time 2024/11/23-18:15:14.581351) EVENT_LOG_v1 {"time_micros": 1732356914581293, "job": 32861, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [274264], "files_L3": [274286, 274287, 274221, 274222, 274223, 274224, 274225, 274226, 274227, 274228, 274229, 274230, 274231, 274232, 274233, 274234, 274235, 274236, 274237, 274288], "score": 0.999441, "input_data_size": 754639284, "oldest_snapshot_seqno": 4175758501} 2024/11/23-18:16:20.873308 139957576050240 (Original Log Time 2024/11/23-18:16:20.871611) [compaction_job.cc:2022] [default] [JOB 32861] Compacted 1@2 + 20@3 files to L3 => 728586441 bytes 2024/11/23-18:16:20.873310 139957576050240 (Original Log Time 2024/11/23-18:16:20.873230) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.4 rd, 11.0 wr, level 3, files in(1, 20) out(20 +0 blob) MB in(27.2, 692.5 +0.0 blob) out(694.8 +0.0 blob), read-write-amplify(52.1) write-amplify(25.6) OK, records in: 17968663, records dropped: 72111 output_compression: Snappy 2024/11/23-18:16:20.873313 139957576050240 (Original Log Time 2024/11/23-18:16:20.873278) EVENT_LOG_v1 {"time_micros": 1732356980873255, "cf": "default", "job": 32861, "event": "compaction_finished", "compaction_time_micros": 66172712, "compaction_time_cpu_micros": 27872014, "output_level": 3, "num_output_files": 20, "total_output_size": 728586441, "num_input_records": 17968663, "num_output_records": 17896552, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 525073893, "file_range_sync_nanos": 0, "file_fsync_nanos": 13628, "file_prepare_write_nanos": 63063, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-18:16:21.013497 139957576050240 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732356914,"executesMs":66422,"compactionJobId":32861,"attempt":0,"compactionInputRawBytes":2596210178,"compactionInputZipBytes":754639284,"compactionOutputRawBytes":0,"compactionOutputZipBytes":728586441,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8e7364c86d80a1d3e1af27bdb42023ca-fb9e26d08e8eba49-00"}}, response = 2024/11/23-18:16:21.013583 139957576050240 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32861/att-00: olev 3, work 66.422 s, result 7.803 ms, install 1.717 ms, input{raw 2.418 GiB zip 719.680 MiB}