2024/11/24-03:39:34.364089 139957332661824 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00: fork to child time = 0.007881 sec 2024/11/24-03:39:34.364846 139957332661824 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00 2024/11/24-03:39:34.370652 139957332661824 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00 2024/11/24-03:39:34.371559 139957332661824 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-03:39:34.371824 139957332661824 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-03:39:34.429190 139957332661824 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40948/att-00/MANIFEST-206347 2024/11/24-03:39:34.459607 139957332661824 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40948/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 290647, last_sequence is 4421866301, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 290490 2024/11/24-03:39:34.459617 139957332661824 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-03:39:34.459754 139957332661824 [version_set.cc:5721] Creating manifest 290647 2024/11/24-03:39:34.493968 139957332661824 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-03:39:34.515190 139957332661824 [compaction_job.cc:2437] [default] [JOB 40948] Compacting 1@2 + 20@3 files to L3, score 0.66, subcompactions 1 : 1 2024/11/24-03:39:34.515206 139957332661824 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 30669 Base level 2, inputs: [290635(27MB)], [290585(35MB) 290586(34MB) 290587(33MB) 290588(34MB) 290589(30MB) 290590(35MB) 290591(32MB) 290592(35MB) 290593(33MB) 290594(35MB) 290595(33MB) 290596(35MB) 290597(33MB) 290598(11MB) 290550(33MB) 290551(34MB) 290552(33MB) 290553(35MB) 290554(33MB) 290555(35MB)] 2024/11/24-03:39:34.521816 139957332661824 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:39:34.521846 139957332661824 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:39:34.521865 139957332661824 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:39:34.521870 139957332661824 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:39:34.521882 139957332661824 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:39:35.277367 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290648.sst) /dev/shm/ToplingTemp/Topling-260646-lHYlp3 2024/11/24-03:39:35.905564 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290648.sst): first pass time = 0.63's, 213.675'MB/sec 2024/11/24-03:39:37.382377 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290648.sst): old prealloc_size = 147639500, real_size = 33346400 2024/11/24-03:39:37.382936 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290648.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926906 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.80 avg-zval = 33.82 2024/11/24-03:39:37.383745 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290648.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:39:38.246556 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290648.sst): second pass time = 1.48's, 73.378'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.331'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3328.321'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 287.478'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35610.640'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.378'MB/sec zip pipeline throughput = 73.378'MB/sec entries = 926906 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 33.82 usrkeys = 926906 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 33.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926906 bytes = 3244192 } seq expand size = 6488342 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5131 value = 3.4533 dict = 1.00 all = 3.5844 } Zip/UnZip{ index = 0.0951 value = 0.2896 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 2251.440326 GB avg = 0.200 KB (by entry num) total key len = 306.908916 GB avg = 0.027 KB total ukey len = 237.373265 GB avg = 0.015 KB total ukey num = 15.739588069 Billion total entry num = 11.280902017 Billion write speed all = 37.920553051 MB/sec (with seq num) write speed all = 36.582884099 MB/sec (without seq num) 2024/11/24-03:39:38.291791 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290648: 926906 keys, 37445008 bytes, temperature: kWarm 2024/11/24-03:39:38.291887 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390778291828, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290648, "file_size": 37445008, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4421771110, "table_properties": {"data_size": 31351184, "index_size": 1763328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953368, "raw_average_key_size": 28, "raw_value_size": 108264408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926906, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390775, "oldest_key_time": 0, "file_creation_time": 1732390775, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290648, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:39:38.327721 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290649.sst) /dev/shm/ToplingTemp/Topling-260646-04bxg1 2024/11/24-03:39:39.235661 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290649.sst): first pass time = 0.91's, 147.832'MB/sec 2024/11/24-03:39:40.654397 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290649.sst): old prealloc_size = 147639500, real_size = 32111216 2024/11/24-03:39:40.655093 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290649.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926477 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 116.87 avg-zval = 32.51 2024/11/24-03:39:40.655993 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290649.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:39:41.536558 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290649.sst): second pass time = 1.42's, 76.372'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.046'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2594.663'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.15's, 237.450'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34047.963'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.372'MB/sec zip pipeline throughput = 76.372'MB/sec entries = 926477 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 32.51 usrkeys = 926477 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 32.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926477 bytes = 3242688 } seq expand size = 6485339 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5140 value = 3.5952 dict = 1.00 all = 3.7068 } Zip/UnZip{ index = 0.0951 value = 0.2781 dict = 1.00 all = 0.2698 } ---------------------------- total value len = 2251.555088 GB avg = 0.200 KB (by entry num) total key len = 306.934857 GB avg = 0.027 KB total ukey len = 237.391795 GB avg = 0.015 KB total ukey num = 15.740514546 Billion total entry num = 11.281828494 Billion write speed all = 37.920789372 MB/sec (with seq num) write speed all = 36.583075794 MB/sec (without seq num) 2024/11/24-03:39:41.580214 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290649: 926477 keys, 36208588 bytes, temperature: kWarm 2024/11/24-03:39:41.580285 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390781580249, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290649, "file_size": 36208588, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4421771468, "table_properties": {"data_size": 30117024, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941356, "raw_average_key_size": 28, "raw_value_size": 108276414, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926477, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390778, "oldest_key_time": 0, "file_creation_time": 1732390778, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290649, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:39:41.615844 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290650.sst) /dev/shm/ToplingTemp/Topling-260646-IHF3J3 2024/11/24-03:39:42.551797 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290650.sst): first pass time = 0.94's, 143.407'MB/sec 2024/11/24-03:39:43.882618 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290650.sst): old prealloc_size = 147639500, real_size = 31330048 2024/11/24-03:39:43.883197 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290650.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922033 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.57 avg-zval = 31.80 2024/11/24-03:39:43.883989 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290650.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:39:44.846129 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290650.sst): second pass time = 1.33's, 81.504'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 36.756'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3208.267'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.280'MB/sec write SST data time = 0.14's, 252.982'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34410.761'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.504'MB/sec zip pipeline throughput = 81.504'MB/sec entries = 922033 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.57 avg-zval = 31.80 usrkeys = 922033 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.57 avg-zval = 31.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922033 bytes = 3227136 } seq expand size = 6454231 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.3769 value = 3.6969 dict = 1.00 all = 3.7900 } Zip/UnZip{ index = 0.0964 value = 0.2705 dict = 1.00 all = 0.2639 } ---------------------------- total value len = 2251.669943 GB avg = 0.200 KB (by entry num) total key len = 306.960674 GB avg = 0.027 KB total ukey len = 237.410236 GB avg = 0.015 KB total ukey num = 15.741436579 Billion total entry num = 11.282750527 Billion write speed all = 37.921014210 MB/sec (with seq num) write speed all = 36.583256926 MB/sec (without seq num) 2024/11/24-03:39:44.889267 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290650: 922033 keys, 35413994 bytes, temperature: kWarm 2024/11/24-03:39:44.889335 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390784889299, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290650, "file_size": 35413994, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4285423486, "largest_seqno": 4421770416, "table_properties": {"data_size": 29322288, "index_size": 1777088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25816924, "raw_average_key_size": 28, "raw_value_size": 108400826, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922033, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390781, "oldest_key_time": 0, "file_creation_time": 1732390781, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290650, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:39:44.926107 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290651.sst) /dev/shm/ToplingTemp/Topling-260646-mpr9B0 2024/11/24-03:39:45.935908 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290651.sst): first pass time = 1.01's, 132.920'MB/sec 2024/11/24-03:39:47.291895 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290651.sst): old prealloc_size = 147639500, real_size = 32551504 2024/11/24-03:39:47.292458 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290651.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928679 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.53 avg-zval = 32.89 2024/11/24-03:39:47.293246 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290651.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:39:48.235133 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290651.sst): second pass time = 1.36's, 79.855'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 38.866'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3285.896'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.13's, 290.609'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32529.910'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.855'MB/sec zip pipeline throughput = 79.855'MB/sec entries = 928679 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.53 avg-zval = 32.89 usrkeys = 928679 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.53 avg-zval = 32.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928679 bytes = 3250384 } seq expand size = 6500753 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4445 value = 3.5433 dict = 1.00 all = 3.6613 } Zip/UnZip{ index = 0.0957 value = 0.2822 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 2251.784658 GB avg = 0.200 KB (by entry num) total key len = 306.986677 GB avg = 0.027 KB total ukey len = 237.428809 GB avg = 0.015 KB total ukey num = 15.742365258 Billion total entry num = 11.283679206 Billion write speed all = 37.921195078 MB/sec (with seq num) write speed all = 36.583394878 MB/sec (without seq num) 2024/11/24-03:39:48.279469 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290651: 928679 keys, 36658720 bytes, temperature: kWarm 2024/11/24-03:39:48.279537 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390788279504, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290651, "file_size": 36658720, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4421771525, "table_properties": {"data_size": 30540864, "index_size": 1778304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003012, "raw_average_key_size": 28, "raw_value_size": 108214774, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928679, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390784, "oldest_key_time": 0, "file_creation_time": 1732390784, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290651, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:39:48.320597 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290652.sst) /dev/shm/ToplingTemp/Topling-260646-P90xR2 2024/11/24-03:39:49.181135 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290652.sst): first pass time = 0.86's, 155.975'MB/sec 2024/11/24-03:39:50.452881 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290652.sst): old prealloc_size = 147639500, real_size = 27633952 2024/11/24-03:39:50.453491 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290652.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0256 GB avg-val = 114.00 avg-zval = 27.07 2024/11/24-03:39:50.454256 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290652.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:39:51.392933 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290652.sst): second pass time = 1.27's, 84.776'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 33.685'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3089.431'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.292'MB/sec write SST data time = 0.13's, 244.628'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 36850.699'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.776'MB/sec zip pipeline throughput = 84.776'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.07 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0256 GB dict = 1.07 MB all = 0.0317 GB } UnZip/Zip{ index = 10.4439 value = 4.2111 dict = 1.00 all = 4.2387 } Zip/UnZip{ index = 0.0957 value = 0.2375 dict = 1.00 all = 0.2359 } ---------------------------- total value len = 2251.899027 GB avg = 0.200 KB (by entry num) total key len = 307.013143 GB avg = 0.027 KB total ukey len = 237.447713 GB avg = 0.015 KB total ukey num = 15.743310454 Billion total entry num = 11.284624402 Billion write speed all = 37.921507567 MB/sec (with seq num) write speed all = 36.583657915 MB/sec (without seq num) 2024/11/24-03:39:51.433794 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290652: 945196 keys, 31665136 bytes, temperature: kWarm 2024/11/24-03:39:51.433860 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390791433826, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290652, "file_size": 31665136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4283009896, "largest_seqno": 4383839606, "table_properties": {"data_size": 25587408, "index_size": 1810048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390788, "oldest_key_time": 0, "file_creation_time": 1732390788, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290652, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:39:51.468909 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290653.sst) /dev/shm/ToplingTemp/Topling-260646-vnLx11 2024/11/24-03:39:52.500495 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290653.sst): first pass time = 1.03's, 130.113'MB/sec 2024/11/24-03:39:53.853765 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290653.sst): old prealloc_size = 147639500, real_size = 32982656 2024/11/24-03:39:53.854344 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290653.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928890 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.49 avg-zval = 33.35 2024/11/24-03:39:53.855122 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290653.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:39:54.825248 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290653.sst): second pass time = 1.35's, 80.022'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.176'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3172.390'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.275'MB/sec write SST data time = 0.13's, 293.183'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32026.709'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.022'MB/sec zip pipeline throughput = 80.022'MB/sec entries = 928890 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 33.35 usrkeys = 928890 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 33.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928890 bytes = 3251136 } seq expand size = 6502230 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5112 value = 3.4925 dict = 1.00 all = 3.6190 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 2252.013738 GB avg = 0.200 KB (by entry num) total key len = 307.039152 GB avg = 0.027 KB total ukey len = 237.466291 GB avg = 0.015 KB total ukey num = 15.744239344 Billion total entry num = 11.285553292 Billion write speed all = 37.921664072 MB/sec (with seq num) write speed all = 36.583772347 MB/sec (without seq num) 2024/11/24-03:39:54.879478 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290653: 928890 keys, 37086612 bytes, temperature: kWarm 2024/11/24-03:39:54.879542 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390794879509, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290653, "file_size": 37086612, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4421769607, "table_properties": {"data_size": 30982832, "index_size": 1767424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26008920, "raw_average_key_size": 28, "raw_value_size": 108208940, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928890, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390791, "oldest_key_time": 0, "file_creation_time": 1732390791, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290653, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:39:54.913479 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290654.sst) /dev/shm/ToplingTemp/Topling-260646-lstc0Z 2024/11/24-03:39:55.890418 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290654.sst): first pass time = 0.98's, 137.391'MB/sec 2024/11/24-03:39:57.224433 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290654.sst): old prealloc_size = 147639500, real_size = 29744512 2024/11/24-03:39:57.224970 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290654.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933361 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0277 GB avg-val = 115.80 avg-zval = 29.70 2024/11/24-03:39:57.225734 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290654.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:39:58.156953 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290654.sst): second pass time = 1.33's, 81.073'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 36.186'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3393.742'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.13's, 261.473'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32706.057'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.073'MB/sec zip pipeline throughput = 81.073'MB/sec entries = 933361 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.80 avg-zval = 29.70 usrkeys = 933361 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.80 avg-zval = 29.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933361 bytes = 3150112 } seq expand size = 6533527 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0277 GB dict = 1.08 MB all = 0.0337 GB } UnZip/Zip{ index = 10.4442 value = 3.8986 dict = 1.00 all = 3.9776 } Zip/UnZip{ index = 0.0957 value = 0.2565 dict = 1.00 all = 0.2514 } ---------------------------- total value len = 2252.128355 GB avg = 0.200 KB (by entry num) total key len = 307.065286 GB avg = 0.027 KB total ukey len = 237.484958 GB avg = 0.015 KB total ukey num = 15.745172705 Billion total entry num = 11.286486653 Billion write speed all = 37.921877558 MB/sec (with seq num) write speed all = 36.583941239 MB/sec (without seq num) 2024/11/24-03:39:58.204220 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290654: 933361 keys, 33743518 bytes, temperature: kWarm 2024/11/24-03:39:58.204295 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390798204258, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290654, "file_size": 33743518, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4312598249, "largest_seqno": 4421769061, "table_properties": {"data_size": 27723632, "index_size": 1787328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26134108, "raw_average_key_size": 28, "raw_value_size": 108083674, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933361, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390794, "oldest_key_time": 0, "file_creation_time": 1732390794, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290654, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:39:58.241054 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290655.sst) /dev/shm/ToplingTemp/Topling-260646-SPsVm3 2024/11/24-03:39:59.276916 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290655.sst): first pass time = 1.04's, 129.576'MB/sec 2024/11/24-03:40:00.639032 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290655.sst): old prealloc_size = 147639500, real_size = 32584496 2024/11/24-03:40:00.639588 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290655.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932955 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.86 avg-zval = 32.76 2024/11/24-03:40:00.640397 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290655.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:01.611510 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290655.sst): second pass time = 1.36's, 79.406'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 37.740'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3356.883'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.13's, 280.245'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36540.628'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.406'MB/sec zip pipeline throughput = 79.406'MB/sec entries = 932955 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.86 avg-zval = 32.76 usrkeys = 932955 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.86 avg-zval = 32.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932955 bytes = 3265360 } seq expand size = 6530685 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4438 value = 3.5366 dict = 1.00 all = 3.6571 } Zip/UnZip{ index = 0.0958 value = 0.2828 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 2252.242981 GB avg = 0.200 KB (by entry num) total key len = 307.091408 GB avg = 0.027 KB total ukey len = 237.503617 GB avg = 0.015 KB total ukey num = 15.746105660 Billion total entry num = 11.287419608 Billion write speed all = 37.922021960 MB/sec (with seq num) write speed all = 36.584043535 MB/sec (without seq num) 2024/11/24-03:40:01.662555 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290655: 932955 keys, 36700328 bytes, temperature: kWarm 2024/11/24-03:40:01.662615 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390801662582, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290655, "file_size": 36700328, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4421771434, "table_properties": {"data_size": 30564448, "index_size": 1786624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26122740, "raw_average_key_size": 28, "raw_value_size": 108095130, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932955, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390798, "oldest_key_time": 0, "file_creation_time": 1732390798, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290655, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:01.700224 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290656.sst) /dev/shm/ToplingTemp/Topling-260646-e2dUA1 2024/11/24-03:40:02.722620 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290656.sst): first pass time = 1.02's, 131.283'MB/sec 2024/11/24-03:40:04.064662 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290656.sst): old prealloc_size = 147639500, real_size = 31131392 2024/11/24-03:40:04.065283 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290656.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922851 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.44 avg-zval = 31.57 2024/11/24-03:40:04.066050 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290656.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:05.000825 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290656.sst): second pass time = 1.34's, 80.801'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 37.497'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2959.820'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.287'MB/sec write SST data time = 0.13's, 265.170'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33896.645'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.801'MB/sec zip pipeline throughput = 80.801'MB/sec entries = 922851 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.44 avg-zval = 31.57 usrkeys = 922851 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.44 avg-zval = 31.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922851 bytes = 3114640 } seq expand size = 6459957 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4459 value = 3.7200 dict = 1.00 all = 3.8236 } Zip/UnZip{ index = 0.0957 value = 0.2688 dict = 1.00 all = 0.2615 } ---------------------------- total value len = 2252.357819 GB avg = 0.200 KB (by entry num) total key len = 307.117248 GB avg = 0.027 KB total ukey len = 237.522074 GB avg = 0.015 KB total ukey num = 15.747028511 Billion total entry num = 11.288342459 Billion write speed all = 37.922201949 MB/sec (with seq num) write speed all = 36.584181328 MB/sec (without seq num) 2024/11/24-03:40:05.047005 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290656: 922851 keys, 35102210 bytes, temperature: kWarm 2024/11/24-03:40:05.047078 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390805047041, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290656, "file_size": 35102210, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4319325603, "largest_seqno": 4421769074, "table_properties": {"data_size": 29133552, "index_size": 1766912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25839828, "raw_average_key_size": 28, "raw_value_size": 108377934, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922851, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390801, "oldest_key_time": 0, "file_creation_time": 1732390801, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290656, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:05.090593 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290657.sst) /dev/shm/ToplingTemp/Topling-260646-e18kX2 2024/11/24-03:40:05.992553 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290657.sst): first pass time = 0.90's, 148.813'MB/sec 2024/11/24-03:40:07.351727 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290657.sst): old prealloc_size = 147639500, real_size = 32779392 2024/11/24-03:40:07.352449 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290657.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931442 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.10 avg-zval = 33.04 2024/11/24-03:40:07.354861 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290657.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:08.234075 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290657.sst): second pass time = 1.35's, 80.046'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.810'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2543.305'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.15's, 249.808'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33677.915'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.046'MB/sec zip pipeline throughput = 80.046'MB/sec entries = 931442 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 33.04 usrkeys = 931442 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 33.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931442 bytes = 3260064 } seq expand size = 6520094 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5119 value = 3.5139 dict = 1.00 all = 3.6383 } Zip/UnZip{ index = 0.0951 value = 0.2846 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 2252.472477 GB avg = 0.200 KB (by entry num) total key len = 307.143329 GB avg = 0.027 KB total ukey len = 237.540703 GB avg = 0.015 KB total ukey num = 15.747959953 Billion total entry num = 11.289273901 Billion write speed all = 37.922470500 MB/sec (with seq num) write speed all = 36.584403575 MB/sec (without seq num) 2024/11/24-03:40:08.283319 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290657: 931442 keys, 36890260 bytes, temperature: kWarm 2024/11/24-03:40:08.283393 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390808283356, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290657, "file_size": 36890260, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4421771327, "table_properties": {"data_size": 30774192, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080376, "raw_average_key_size": 28, "raw_value_size": 108137480, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931442, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390805, "oldest_key_time": 0, "file_creation_time": 1732390805, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290657, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:08.319707 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290658.sst) /dev/shm/ToplingTemp/Topling-260646-Vc4tM0 2024/11/24-03:40:09.205152 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290658.sst): first pass time = 0.89's, 151.589'MB/sec 2024/11/24-03:40:10.627415 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290658.sst): old prealloc_size = 147639500, real_size = 31081200 2024/11/24-03:40:10.628017 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290658.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922276 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.53 avg-zval = 31.54 2024/11/24-03:40:10.628792 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290658.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:11.450882 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290658.sst): second pass time = 1.42's, 76.257'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.566'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3064.467'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 273.757'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34359.404'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.257'MB/sec zip pipeline throughput = 76.257'MB/sec entries = 922276 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.53 avg-zval = 31.54 usrkeys = 922276 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.53 avg-zval = 31.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922276 bytes = 3112704 } seq expand size = 6455932 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4443 value = 3.7269 dict = 1.00 all = 3.8292 } Zip/UnZip{ index = 0.0957 value = 0.2683 dict = 1.00 all = 0.2611 } ---------------------------- total value len = 2252.587327 GB avg = 0.200 KB (by entry num) total key len = 307.169152 GB avg = 0.027 KB total ukey len = 237.559149 GB avg = 0.015 KB total ukey num = 15.748882229 Billion total entry num = 11.290196177 Billion write speed all = 37.922747309 MB/sec (with seq num) write speed all = 36.584634844 MB/sec (without seq num) 2024/11/24-03:40:11.496133 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290658: 922276 keys, 35050728 bytes, temperature: kWarm 2024/11/24-03:40:11.496206 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390811496168, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290658, "file_size": 35050728, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4322599538, "largest_seqno": 4421769093, "table_properties": {"data_size": 29084384, "index_size": 1766080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25823728, "raw_average_key_size": 28, "raw_value_size": 108394044, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922276, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390808, "oldest_key_time": 0, "file_creation_time": 1732390808, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290658, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:11.542583 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290659.sst) /dev/shm/ToplingTemp/Topling-260646-MhvMh2 2024/11/24-03:40:12.715173 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290659.sst): first pass time = 1.17's, 114.466'MB/sec 2024/11/24-03:40:14.066928 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290659.sst): old prealloc_size = 147639500, real_size = 33076528 2024/11/24-03:40:14.067513 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290659.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928754 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.51 avg-zval = 33.46 2024/11/24-03:40:14.069584 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290659.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:14.940796 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290659.sst): second pass time = 1.35's, 80.095'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.548'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3161.918'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 286.813'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30030.550'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.095'MB/sec zip pipeline throughput = 80.095'MB/sec entries = 928754 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.51 avg-zval = 33.46 usrkeys = 928754 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.51 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928754 bytes = 3250656 } seq expand size = 6501278 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5146 value = 3.4820 dict = 1.00 all = 3.6099 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 2252.702041 GB avg = 0.200 KB (by entry num) total key len = 307.195157 GB avg = 0.027 KB total ukey len = 237.577724 GB avg = 0.015 KB total ukey num = 15.749810983 Billion total entry num = 11.291124931 Billion write speed all = 37.922871333 MB/sec (with seq num) write speed all = 36.584717978 MB/sec (without seq num) 2024/11/24-03:40:14.987964 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290659: 928754 keys, 37180144 bytes, temperature: kWarm 2024/11/24-03:40:14.988051 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390814988014, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290659, "file_size": 37180144, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4421770796, "table_properties": {"data_size": 31077536, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005112, "raw_average_key_size": 28, "raw_value_size": 108212700, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928754, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390811, "oldest_key_time": 0, "file_creation_time": 1732390811, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290659, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:15.029073 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290660.sst) /dev/shm/ToplingTemp/Topling-260646-2eayD3 2024/11/24-03:40:16.167443 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290660.sst): first pass time = 1.14's, 117.908'MB/sec 2024/11/24-03:40:17.500372 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290660.sst): old prealloc_size = 147639500, real_size = 30999744 2024/11/24-03:40:17.500926 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290660.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924610 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 117.16 avg-zval = 31.35 2024/11/24-03:40:17.503212 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290660.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:18.340812 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290660.sst): second pass time = 1.33's, 81.355'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.616'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3363.302'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 261.984'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 29477.936'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.355'MB/sec zip pipeline throughput = 81.355'MB/sec entries = 924610 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.16 avg-zval = 31.35 usrkeys = 924610 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.16 avg-zval = 31.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924610 bytes = 3120576 } seq expand size = 6472270 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.3790 value = 3.7372 dict = 1.00 all = 3.8376 } Zip/UnZip{ index = 0.0963 value = 0.2676 dict = 1.00 all = 0.2606 } ---------------------------- total value len = 2252.816842 GB avg = 0.200 KB (by entry num) total key len = 307.221047 GB avg = 0.027 KB total ukey len = 237.596216 GB avg = 0.015 KB total ukey num = 15.750735593 Billion total entry num = 11.292049541 Billion write speed all = 37.923045416 MB/sec (with seq num) write speed all = 36.584849885 MB/sec (without seq num) 2024/11/24-03:40:18.385155 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290660: 924610 keys, 34974716 bytes, temperature: kWarm 2024/11/24-03:40:18.385226 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390818385190, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290660, "file_size": 34974716, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4333344670, "largest_seqno": 4421769111, "table_properties": {"data_size": 28986736, "index_size": 1781696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25889080, "raw_average_key_size": 28, "raw_value_size": 108328712, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924610, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390815, "oldest_key_time": 0, "file_creation_time": 1732390815, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290660, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:18.420222 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290661.sst) /dev/shm/ToplingTemp/Topling-260646-Z2gQr2 2024/11/24-03:40:19.514546 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290661.sst): first pass time = 1.09's, 122.653'MB/sec 2024/11/24-03:40:20.887076 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290661.sst): old prealloc_size = 147639500, real_size = 32880832 2024/11/24-03:40:20.887593 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290661.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925497 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.02 avg-zval = 33.36 2024/11/24-03:40:20.888387 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290661.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:21.740954 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290661.sst): second pass time = 1.37's, 78.945'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.304'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3566.064'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 279.281'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35131.959'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.945'MB/sec zip pipeline throughput = 78.945'MB/sec entries = 925497 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.02 avg-zval = 33.36 usrkeys = 925497 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.02 avg-zval = 33.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925497 bytes = 3239248 } seq expand size = 6478479 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4430 value = 3.5076 dict = 1.00 all = 3.6300 } Zip/UnZip{ index = 0.0958 value = 0.2851 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 2252.931624 GB avg = 0.199 KB (by entry num) total key len = 307.246960 GB avg = 0.027 KB total ukey len = 237.614726 GB avg = 0.015 KB total ukey num = 15.751661090 Billion total entry num = 11.292975038 Billion write speed all = 37.923219503 MB/sec (with seq num) write speed all = 36.584981698 MB/sec (without seq num) 2024/11/24-03:40:21.787935 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290661: 925497 keys, 36974922 bytes, temperature: kWarm 2024/11/24-03:40:21.788015 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390821787970, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290661, "file_size": 36974922, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4421770543, "table_properties": {"data_size": 30876784, "index_size": 1772480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25913916, "raw_average_key_size": 28, "raw_value_size": 108303830, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925497, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390818, "oldest_key_time": 0, "file_creation_time": 1732390818, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290661, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:21.823663 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290662.sst) /dev/shm/ToplingTemp/Topling-260646-llmsw0 2024/11/24-03:40:22.734251 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290662.sst): first pass time = 0.91's, 147.402'MB/sec 2024/11/24-03:40:24.137147 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290662.sst): old prealloc_size = 147639500, real_size = 30449152 2024/11/24-03:40:24.137699 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290662.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927880 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.65 avg-zval = 30.65 2024/11/24-03:40:24.139719 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290662.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:25.155663 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290662.sst): second pass time = 1.40's, 77.192'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 33.807'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3355.868'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.262'MB/sec write SST data time = 0.13's, 259.812'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35839.395'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.192'MB/sec zip pipeline throughput = 77.192'MB/sec entries = 927880 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.65 avg-zval = 30.65 usrkeys = 927880 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.65 avg-zval = 30.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927880 bytes = 3131616 } seq expand size = 6495160 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.4453 value = 3.8058 dict = 1.00 all = 3.8980 } Zip/UnZip{ index = 0.0957 value = 0.2628 dict = 1.00 all = 0.2565 } ---------------------------- total value len = 2253.046356 GB avg = 0.199 KB (by entry num) total key len = 307.272941 GB avg = 0.027 KB total ukey len = 237.633283 GB avg = 0.015 KB total ukey num = 15.752588970 Billion total entry num = 11.293902918 Billion write speed all = 37.923385635 MB/sec (with seq num) write speed all = 36.585105566 MB/sec (without seq num) 2024/11/24-03:40:25.199129 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290662: 927880 keys, 34432840 bytes, temperature: kWarm 2024/11/24-03:40:25.199193 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390825199158, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290662, "file_size": 34432840, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4347027166, "largest_seqno": 4421769125, "table_properties": {"data_size": 28440368, "index_size": 1776640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980640, "raw_average_key_size": 28, "raw_value_size": 108237144, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927880, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390821, "oldest_key_time": 0, "file_creation_time": 1732390821, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290662, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:25.238712 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290663.sst) /dev/shm/ToplingTemp/Topling-260646-xoT9H3 2024/11/24-03:40:26.210599 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290663.sst): first pass time = 0.97's, 138.107'MB/sec 2024/11/24-03:40:27.540920 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290663.sst): old prealloc_size = 147639500, real_size = 32427424 2024/11/24-03:40:27.541519 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290663.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922248 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0304 GB avg-val = 117.53 avg-zval = 33.00 2024/11/24-03:40:27.542306 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290663.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:28.368942 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290663.sst): second pass time = 1.33's, 81.530'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.099'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3065.454'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 287.693'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35415.307'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.530'MB/sec zip pipeline throughput = 81.530'MB/sec entries = 922248 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.53 avg-zval = 33.00 usrkeys = 922248 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.53 avg-zval = 33.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922248 bytes = 3227888 } seq expand size = 6455736 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4459 value = 3.5620 dict = 1.00 all = 3.6758 } Zip/UnZip{ index = 0.0957 value = 0.2807 dict = 1.00 all = 0.2720 } ---------------------------- total value len = 2253.161207 GB avg = 0.199 KB (by entry num) total key len = 307.298764 GB avg = 0.027 KB total ukey len = 237.651728 GB avg = 0.015 KB total ukey num = 15.753511218 Billion total entry num = 11.294825166 Billion write speed all = 37.923664311 MB/sec (with seq num) write speed all = 36.585338657 MB/sec (without seq num) 2024/11/24-03:40:28.414630 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290663: 922248 keys, 36513792 bytes, temperature: kWarm 2024/11/24-03:40:28.414698 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390828414662, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290663, "file_size": 36513792, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4421771561, "table_properties": {"data_size": 30430928, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25822944, "raw_average_key_size": 28, "raw_value_size": 108394816, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922248, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390825, "oldest_key_time": 0, "file_creation_time": 1732390825, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290663, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:28.451225 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290664.sst) /dev/shm/ToplingTemp/Topling-260646-jsfoC3 2024/11/24-03:40:29.472176 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290664.sst): first pass time = 1.02's, 131.468'MB/sec 2024/11/24-03:40:30.796181 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290664.sst): old prealloc_size = 147639500, real_size = 30194320 2024/11/24-03:40:30.796759 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290664.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930979 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0282 GB avg-val = 116.17 avg-zval = 30.28 2024/11/24-03:40:30.798751 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290664.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:31.616577 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290664.sst): second pass time = 1.32's, 81.735'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.816'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3218.145'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 263.138'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33653.593'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.735'MB/sec zip pipeline throughput = 81.735'MB/sec entries = 930979 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.17 avg-zval = 30.28 usrkeys = 930979 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.17 avg-zval = 30.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930979 bytes = 3258448 } seq expand size = 6516853 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0282 GB dict = 1.08 MB all = 0.0343 GB } UnZip/Zip{ index = 10.5117 value = 3.8365 dict = 1.00 all = 3.9126 } Zip/UnZip{ index = 0.0951 value = 0.2607 dict = 1.00 all = 0.2556 } ---------------------------- total value len = 2253.275874 GB avg = 0.199 KB (by entry num) total key len = 307.324831 GB avg = 0.027 KB total ukey len = 237.670348 GB avg = 0.015 KB total ukey num = 15.754442197 Billion total entry num = 11.295756145 Billion write speed all = 37.923924567 MB/sec (with seq num) write speed all = 36.585552980 MB/sec (without seq num) 2024/11/24-03:40:31.659240 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290664: 930979 keys, 34304436 bytes, temperature: kWarm 2024/11/24-03:40:31.659311 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390831659275, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290664, "file_size": 34304436, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4421771082, "table_properties": {"data_size": 28190080, "index_size": 1771328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26067412, "raw_average_key_size": 28, "raw_value_size": 108150430, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930979, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390828, "oldest_key_time": 0, "file_creation_time": 1732390828, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290664, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:31.695544 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290665.sst) /dev/shm/ToplingTemp/Topling-260646-mfeGY3 2024/11/24-03:40:32.683325 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290665.sst): first pass time = 0.99's, 135.883'MB/sec 2024/11/24-03:40:33.988837 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290665.sst): old prealloc_size = 147639500, real_size = 31424400 2024/11/24-03:40:33.989432 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290665.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.02 2024/11/24-03:40:33.990164 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290665.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:34.808328 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290665.sst): second pass time = 1.30's, 83.140'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.182'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3068.314'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 275.116'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33916.101'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.140'MB/sec zip pipeline throughput = 83.140'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.02 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4443 value = 3.6854 dict = 1.00 all = 3.7929 } Zip/UnZip{ index = 0.0957 value = 0.2713 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2253.390787 GB avg = 0.199 KB (by entry num) total key len = 307.350572 GB avg = 0.027 KB total ukey len = 237.688734 GB avg = 0.015 KB total ukey num = 15.755361497 Billion total entry num = 11.296675445 Billion write speed all = 37.924214976 MB/sec (with seq num) write speed all = 36.585797735 MB/sec (without seq num) 2024/11/24-03:40:34.851924 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290665: 919300 keys, 35386226 bytes, temperature: kWarm 2024/11/24-03:40:34.852008 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390834851958, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290665, "file_size": 35386226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4266308125, "largest_seqno": 4374292517, "table_properties": {"data_size": 29433984, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390831, "oldest_key_time": 0, "file_creation_time": 1732390831, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290665, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:34.889747 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290666.sst) /dev/shm/ToplingTemp/Topling-260646-sBQGx0 2024/11/24-03:40:35.724048 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290666.sst): first pass time = 0.83's, 160.882'MB/sec 2024/11/24-03:40:37.064419 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290666.sst): old prealloc_size = 147639500, real_size = 31555024 2024/11/24-03:40:37.064960 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290666.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930858 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0296 GB avg-val = 116.19 avg-zval = 31.75 2024/11/24-03:40:37.065736 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290666.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:40:37.900527 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290666.sst): second pass time = 1.34's, 80.764'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.657'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3438.335'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 278.825'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33149.768'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.764'MB/sec zip pipeline throughput = 80.764'MB/sec entries = 930858 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.19 avg-zval = 31.75 usrkeys = 930858 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.19 avg-zval = 31.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930858 bytes = 3258016 } seq expand size = 6516006 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5160 value = 3.6598 dict = 1.00 all = 3.7633 } Zip/UnZip{ index = 0.0951 value = 0.2732 dict = 1.00 all = 0.2657 } ---------------------------- total value len = 2253.505456 GB avg = 0.199 KB (by entry num) total key len = 307.376636 GB avg = 0.027 KB total ukey len = 237.707351 GB avg = 0.015 KB total ukey num = 15.756292355 Billion total entry num = 11.297606303 Billion write speed all = 37.924562471 MB/sec (with seq num) write speed all = 36.586096238 MB/sec (without seq num) 2024/11/24-03:40:37.946997 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290666: 930858 keys, 35664722 bytes, temperature: kWarm 2024/11/24-03:40:37.947069 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390837947034, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290666, "file_size": 35664722, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4421769153, "table_properties": {"data_size": 29551744, "index_size": 1770368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26064024, "raw_average_key_size": 28, "raw_value_size": 108153808, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930858, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390834, "oldest_key_time": 0, "file_creation_time": 1732390834, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290666, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:37.986106 139957332661824 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290667.sst) /dev/shm/ToplingTemp/Topling-260646-H96rM3 2024/11/24-03:40:38.216993 139957332661824 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290667.sst): first pass time = 0.23's, 251.110'MB/sec 2024/11/24-03:40:38.800096 139957332661824 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290667.sst): old prealloc_size = 147639500, real_size = 15250640 2024/11/24-03:40:38.800385 139957332661824 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290667.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 397030 prefix = 4 raw-key = 0.0064 GB zip-key = 0.0008 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0468 GB zip-val = 0.0144 GB avg-val = 118.00 avg-zval = 36.24 2024/11/24-03:40:38.800722 139957332661824 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290667.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.058 GB 2024/11/24-03:40:39.168115 139957332661824 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00/290667.sst): second pass time = 0.58's, 80.450'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 45.980'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2972.026'MB/sec (index lex order gen) rebuild zvType time = 0.31's, 0.318'MB/sec write SST data time = 0.06's, 302.838'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.476'MB, throughput = 15043.379'MB/sec zip my value time = 0.58's, unzip length = 0.047'GB zip my value throughput = 80.450'MB/sec zip pipeline throughput = 80.450'MB/sec entries = 397030 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 36.24 usrkeys = 397030 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 36.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 397030 bytes = 1290368 } seq expand size = 2779210 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0079 GB value = 0.0468 GB dict = 0.48 MB all = 0.0580 GB } __ZipSize{ index = 0.0008 GB value = 0.0144 GB dict = 0.48 MB all = 0.0169 GB } UnZip/Zip{ index = 10.4315 value = 3.2557 dict = 1.00 all = 3.4258 } Zip/UnZip{ index = 0.0959 value = 0.3072 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 2253.555085 GB avg = 0.199 KB (by entry num) total key len = 307.387753 GB avg = 0.027 KB total ukey len = 237.715292 GB avg = 0.015 KB total ukey num = 15.756689385 Billion total entry num = 11.298003333 Billion write speed all = 37.924750143 MB/sec (with seq num) write speed all = 36.586261998 MB/sec (without seq num) 2024/11/24-03:40:39.194266 139957332661824 [compaction_job.cc:1948] [default] [JOB 40948] Generated table #290667: 397030 keys, 16920611 bytes, temperature: kWarm 2024/11/24-03:40:39.194333 139957332661824 EVENT_LOG_v1 {"time_micros": 1732390839194299, "cf_name": "default", "job": 40948, "event": "table_file_creation", "file_number": 290667, "file_size": 16920611, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4279135470, "largest_seqno": 4325494344, "table_properties": {"data_size": 14389952, "index_size": 761216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 11116840, "raw_average_key_size": 28, "raw_value_size": 46849540, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 397030, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732390837, "oldest_key_time": 0, "file_creation_time": 1732390837, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290667, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:40:39.211712 139957332661824 [compaction_job.cc:703] job-40948: subcompact[0], size: 0.694318 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40948/att-00 [290648,290649,290650,290651,290652,290653,290654,290655,290656,290657,290658,290659,290660,290661,290662,290663,290664,290665,290666,290667] 2024/11/24-03:40:39.325587 139957332661824 (Original Log Time 2024/11/24-03:39:34.515266) EVENT_LOG_v1 {"time_micros": 1732390774515224, "job": 40948, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [290635], "files_L3": [290585, 290586, 290587, 290588, 290589, 290590, 290591, 290592, 290593, 290594, 290595, 290596, 290597, 290598, 290550, 290551, 290552, 290553, 290554, 290555], "score": 0.655986, "input_data_size": 720313453, "oldest_snapshot_seqno": 4421866255} 2024/11/24-03:40:39.325590 139957332661824 (Original Log Time 2024/11/24-03:40:39.323947) [compaction_job.cc:2022] [default] [JOB 40948] Compacted 1@2 + 20@3 files to L3 => 694317511 bytes 2024/11/24-03:40:39.325592 139957332661824 (Original Log Time 2024/11/24-03:40:39.325517) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.1 rd, 10.7 wr, level 3, files in(1, 20) out(20 +0 blob) MB in(27.1, 659.9 +0.0 blob) out(662.2 +0.0 blob), read-write-amplify(49.8) write-amplify(24.4) OK, records in: 18128875, records dropped: 100653 output_compression: Snappy 2024/11/24-03:40:39.325595 139957332661824 (Original Log Time 2024/11/24-03:40:39.325561) EVENT_LOG_v1 {"time_micros": 1732390839325540, "cf": "default", "job": 40948, "event": "compaction_finished", "compaction_time_micros": 64696455, "compaction_time_cpu_micros": 28740627, "output_level": 3, "num_output_files": 20, "total_output_size": 694317511, "num_input_records": 18128875, "num_output_records": 18028222, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 480315893, "file_range_sync_nanos": 0, "file_fsync_nanos": 13901, "file_prepare_write_nanos": 55588, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/24-03:40:39.467949 139957332661824 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732390774,"executesMs":64951,"compactionJobId":40948,"attempt":0,"compactionInputRawBytes":2622397290,"compactionInputZipBytes":720313453,"compactionOutputRawBytes":0,"compactionOutputZipBytes":694317511,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b293bff155332696428c9d72bbd51a84-bb1b41e3e9f88e0a-00"}}, response = 2024/11/24-03:40:39.468057 139957332661824 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40948/att-00: olev 3, work 64.952 s, result 7.772 ms, install 1.663 ms, input{raw 2.442 GiB zip 686.944 MiB}