2024/11/24-05:12:13.560549 139957215163968 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00: fork to child time = 0.008886 sec 2024/11/24-05:12:13.561407 139957215163968 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00 2024/11/24-05:12:13.567053 139957215163968 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00 2024/11/24-05:12:13.567946 139957215163968 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-05:12:13.568212 139957215163968 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-05:12:13.628425 139957215163968 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42006/att-00/MANIFEST-206347 2024/11/24-05:12:13.645110 139957215163968 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42006/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 293099, last_sequence is 4454509562, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 292960 2024/11/24-05:12:13.645119 139957215163968 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-05:12:13.645263 139957215163968 [version_set.cc:5721] Creating manifest 293099 2024/11/24-05:12:13.684408 139957215163968 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-05:12:13.705657 139957215163968 [compaction_job.cc:2437] [default] [JOB 42006] Compacting 1@2 + 22@3 files to L3, score 0.97, subcompactions 1 : 1 2024/11/24-05:12:13.705679 139957215163968 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31492 Base level 2, inputs: [293097(27MB)], [293058(34MB) 293059(31MB) 293060(35MB) 293061(33MB) 293062(33MB) 293063(33MB) 293064(32MB) 293065(33MB) 293066(32MB) 293067(34MB) 293068(33MB) 293069(35MB) 293070(2651KB) 293015(33MB) 293016(35MB) 293017(33MB) 293018(35MB) 293019(33MB) 293020(34MB) 293021(33MB) 293022(32MB) 293023(765KB)] 2024/11/24-05:12:13.712480 139957215163968 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:12:13.712510 139957215163968 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:12:13.712530 139957215163968 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:12:13.712536 139957215163968 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:12:13.712547 139957215163968 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:12:14.361934 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293100.sst) /dev/shm/ToplingTemp/Topling-277024-tFIZbu 2024/11/24-05:12:14.920984 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293100.sst): first pass time = 0.56's, 240.111'MB/sec 2024/11/24-05:12:16.490607 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293100.sst): old prealloc_size = 147639500, real_size = 32119728 2024/11/24-05:12:16.491306 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293100.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920957 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0301 GB avg-val = 117.74 avg-zval = 32.71 2024/11/24-05:12:16.492166 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293100.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:17.351360 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293100.sst): second pass time = 1.56's, 69.481'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.059'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2601.319'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.15's, 234.371'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33799.913'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.481'MB/sec zip pipeline throughput = 69.481'MB/sec entries = 920957 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.74 avg-zval = 32.71 usrkeys = 920957 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.74 avg-zval = 32.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920957 bytes = 3223360 } seq expand size = 6446699 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4426 value = 3.5993 dict = 1.00 all = 3.7074 } Zip/UnZip{ index = 0.0958 value = 0.2778 dict = 1.00 all = 0.2697 } ---------------------------- total value len = 2251.440451 GB avg = 0.200 KB (by entry num) total key len = 306.908749 GB avg = 0.027 KB total ukey len = 237.373146 GB avg = 0.015 KB total ukey num = 15.739582120 Billion total entry num = 11.280896068 Billion write speed all = 35.033815964 MB/sec (with seq num) write speed all = 33.797978928 MB/sec (without seq num) 2024/11/24-05:12:17.398343 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293100: 920957 keys, 36202564 bytes, temperature: kWarm 2024/11/24-05:12:17.398434 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396337398379, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293100, "file_size": 36202564, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4454499270, "table_properties": {"data_size": 30125472, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25786796, "raw_average_key_size": 28, "raw_value_size": 108431066, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920957, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396334, "oldest_key_time": 0, "file_creation_time": 1732396334, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293100, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:17.443081 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293101.sst) /dev/shm/ToplingTemp/Topling-277024-se1DQt 2024/11/24-05:12:18.834625 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293101.sst): first pass time = 1.39's, 96.455'MB/sec 2024/11/24-05:12:21.008840 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293101.sst): old prealloc_size = 147639500, real_size = 29586688 2024/11/24-05:12:21.009490 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293101.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935740 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0275 GB avg-val = 115.43 avg-zval = 29.44 2024/11/24-05:12:21.010320 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293101.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:21.866672 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293101.sst): second pass time = 2.17's, 49.697'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 39.161'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2870.833'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.327'MB/sec write SST data time = 0.14's, 236.726'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 26814.372'MB/sec zip my value time = 2.17's, unzip length = 0.108'GB zip my value throughput = 49.697'MB/sec zip pipeline throughput = 72.496'MB/sec entries = 935740 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.43 avg-zval = 29.44 usrkeys = 935740 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.43 avg-zval = 29.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935740 bytes = 3158144 } seq expand size = 6550180 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0275 GB dict = 1.08 MB all = 0.0336 GB } UnZip/Zip{ index = 10.3753 value = 3.9209 dict = 1.00 all = 3.9955 } Zip/UnZip{ index = 0.0964 value = 0.2550 dict = 1.00 all = 0.2503 } ---------------------------- total value len = 2251.555018 GB avg = 0.200 KB (by entry num) total key len = 306.934950 GB avg = 0.027 KB total ukey len = 237.391861 GB avg = 0.015 KB total ukey num = 15.740517860 Billion total entry num = 11.281831808 Billion write speed all = 35.033577424 MB/sec (with seq num) write speed all = 33.797714292 MB/sec (without seq num) 2024/11/24-05:12:21.912563 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293101: 935740 keys, 33592640 bytes, temperature: kWarm 2024/11/24-05:12:21.912628 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396341912596, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293101, "file_size": 33592640, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4333816988, "largest_seqno": 4454499138, "table_properties": {"data_size": 27548784, "index_size": 1803776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26200720, "raw_average_key_size": 28, "raw_value_size": 108017132, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935740, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396337, "oldest_key_time": 0, "file_creation_time": 1732396337, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293101, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:21.954849 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293102.sst) /dev/shm/ToplingTemp/Topling-277024-OxoExu 2024/11/24-05:12:23.320101 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293102.sst): first pass time = 1.37's, 98.313'MB/sec 2024/11/24-05:12:25.668064 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293102.sst): old prealloc_size = 147639500, real_size = 32490080 2024/11/24-05:12:25.668822 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293102.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.82 avg-zval = 32.66 2024/11/24-05:12:25.669676 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293102.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:26.568154 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293102.sst): second pass time = 2.35's, 46.050'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.671'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2423.363'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.16's, 230.228'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35278.701'MB/sec zip my value time = 2.35's, unzip length = 0.108'GB zip my value throughput = 46.050'MB/sec zip pipeline throughput = 68.806'MB/sec entries = 933242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.82 avg-zval = 32.66 usrkeys = 933242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.82 avg-zval = 32.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933242 bytes = 3266368 } seq expand size = 6532694 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5140 value = 3.5460 dict = 1.00 all = 3.6665 } Zip/UnZip{ index = 0.0951 value = 0.2820 dict = 1.00 all = 0.2727 } ---------------------------- total value len = 2251.669638 GB avg = 0.200 KB (by entry num) total key len = 306.961081 GB avg = 0.027 KB total ukey len = 237.410526 GB avg = 0.015 KB total ukey num = 15.741451102 Billion total entry num = 11.282765050 Billion write speed all = 35.033249371 MB/sec (with seq num) write speed all = 33.797363571 MB/sec (without seq num) 2024/11/24-05:12:26.631684 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293102: 933242 keys, 36607016 bytes, temperature: kWarm 2024/11/24-05:12:26.631733 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396346631703, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293102, "file_size": 36607016, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4454498830, "table_properties": {"data_size": 30481360, "index_size": 1775232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26130776, "raw_average_key_size": 28, "raw_value_size": 108087068, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396341, "oldest_key_time": 0, "file_creation_time": 1732396341, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293102, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:26.684273 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293103.sst) /dev/shm/ToplingTemp/Topling-277024-TX4UYp 2024/11/24-05:12:27.838297 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293103.sst): first pass time = 1.15's, 116.308'MB/sec 2024/11/24-05:12:29.178427 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293103.sst): old prealloc_size = 147639500, real_size = 31429344 2024/11/24-05:12:29.179027 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293103.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.02 2024/11/24-05:12:29.179814 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293103.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:30.004728 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293103.sst): second pass time = 1.34's, 81.026'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.831'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3050.381'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 272.418'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36725.223'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.026'MB/sec zip pipeline throughput = 81.026'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.02 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4455 value = 3.6848 dict = 1.00 all = 3.7924 } Zip/UnZip{ index = 0.0957 value = 0.2714 dict = 1.00 all = 0.2637 } ---------------------------- total value len = 2251.784551 GB avg = 0.200 KB (by entry num) total key len = 306.986821 GB avg = 0.027 KB total ukey len = 237.428912 GB avg = 0.015 KB total ukey num = 15.742370402 Billion total entry num = 11.283684350 Billion write speed all = 35.033526735 MB/sec (with seq num) write speed all = 33.797598393 MB/sec (without seq num) 2024/11/24-05:12:30.055481 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293103: 919300 keys, 35391154 bytes, temperature: kWarm 2024/11/24-05:12:30.055548 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396350055516, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293103, "file_size": 35391154, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4318015597, "largest_seqno": 4436940634, "table_properties": {"data_size": 29439120, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396346, "oldest_key_time": 0, "file_creation_time": 1732396346, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293103, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:30.095610 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293104.sst) /dev/shm/ToplingTemp/Topling-277024-JDHWts 2024/11/24-05:12:31.243554 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293104.sst): first pass time = 1.15's, 116.924'MB/sec 2024/11/24-05:12:32.605444 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293104.sst): old prealloc_size = 147639500, real_size = 31136064 2024/11/24-05:12:32.606023 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293104.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936400 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0291 GB avg-val = 115.33 avg-zval = 31.10 2024/11/24-05:12:32.606799 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293104.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:33.458619 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293104.sst): second pass time = 1.36's, 79.377'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.332'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3199.828'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 269.255'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 31978.325'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.377'MB/sec zip pipeline throughput = 79.377'MB/sec entries = 936400 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.33 avg-zval = 31.10 usrkeys = 936400 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.33 avg-zval = 31.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936400 bytes = 3277408 } seq expand size = 6554800 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.5110 value = 3.7087 dict = 1.00 all = 3.8063 } Zip/UnZip{ index = 0.0951 value = 0.2696 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 2251.899104 GB avg = 0.200 KB (by entry num) total key len = 307.013040 GB avg = 0.027 KB total ukey len = 237.447640 GB avg = 0.015 KB total ukey num = 15.743306802 Billion total entry num = 11.284620750 Billion write speed all = 35.033797418 MB/sec (with seq num) write speed all = 33.797824958 MB/sec (without seq num) 2024/11/24-05:12:33.503926 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293104: 936400 keys, 35262326 bytes, temperature: kWarm 2024/11/24-05:12:33.504009 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396353503958, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293104, "file_size": 35262326, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4454499151, "table_properties": {"data_size": 29120048, "index_size": 1781760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26219200, "raw_average_key_size": 28, "raw_value_size": 107998588, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936400, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396350, "oldest_key_time": 0, "file_creation_time": 1732396350, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293104, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:33.544268 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293105.sst) /dev/shm/ToplingTemp/Topling-277024-dhgYRq 2024/11/24-05:12:34.689275 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293105.sst): first pass time = 1.14's, 117.223'MB/sec 2024/11/24-05:12:36.035089 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293105.sst): old prealloc_size = 147639500, real_size = 31405216 2024/11/24-05:12:36.035650 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293105.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.98 2024/11/24-05:12:36.036401 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293105.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:36.856732 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293105.sst): second pass time = 1.35's, 80.646'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.046'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3286.511'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 275.249'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30589.512'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.646'MB/sec zip pipeline throughput = 80.646'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.98 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.3768 value = 3.6893 dict = 1.00 all = 3.7950 } Zip/UnZip{ index = 0.0964 value = 0.2711 dict = 1.00 all = 0.2635 } ---------------------------- total value len = 2252.014016 GB avg = 0.200 KB (by entry num) total key len = 307.038781 GB avg = 0.027 KB total ukey len = 237.466026 GB avg = 0.015 KB total ukey num = 15.744226102 Billion total entry num = 11.285540050 Billion write speed all = 35.034093183 MB/sec (with seq num) write speed all = 33.798077538 MB/sec (without seq num) 2024/11/24-05:12:36.900674 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293105: 919300 keys, 35367218 bytes, temperature: kWarm 2024/11/24-05:12:36.900734 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396356900701, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293105, "file_size": 35367218, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4286064902, "largest_seqno": 4396673044, "table_properties": {"data_size": 29403344, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396353, "oldest_key_time": 0, "file_creation_time": 1732396353, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293105, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:36.942833 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293106.sst) /dev/shm/ToplingTemp/Topling-277024-yDYMxr 2024/11/24-05:12:38.073261 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293106.sst): first pass time = 1.13's, 118.735'MB/sec 2024/11/24-05:12:39.411935 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293106.sst): old prealloc_size = 147639500, real_size = 29963312 2024/11/24-05:12:39.412526 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293106.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931971 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0279 GB avg-val = 116.01 avg-zval = 29.99 2024/11/24-05:12:39.413295 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293106.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:40.239470 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293106.sst): second pass time = 1.34's, 80.845'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 40.895'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3133.772'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 261.839'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32385.075'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.845'MB/sec zip pipeline throughput = 80.845'MB/sec entries = 931971 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.01 avg-zval = 29.99 usrkeys = 931971 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.01 avg-zval = 29.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931971 bytes = 3028928 } seq expand size = 6523797 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0279 GB dict = 1.08 MB all = 0.0338 GB } UnZip/Zip{ index = 10.4455 value = 3.8690 dict = 1.00 all = 3.9661 } Zip/UnZip{ index = 0.0957 value = 0.2585 dict = 1.00 all = 0.2521 } ---------------------------- total value len = 2252.128663 GB avg = 0.200 KB (by entry num) total key len = 307.064876 GB avg = 0.027 KB total ukey len = 237.484666 GB avg = 0.015 KB total ukey num = 15.745158073 Billion total entry num = 11.286472021 Billion write speed all = 35.034397505 MB/sec (with seq num) write speed all = 33.798337031 MB/sec (without seq num) 2024/11/24-05:12:40.282184 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293106: 931971 keys, 33841282 bytes, temperature: kWarm 2024/11/24-05:12:40.282261 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396360282218, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293106, "file_size": 33841282, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4396673045, "largest_seqno": 4454499161, "table_properties": {"data_size": 27945696, "index_size": 1784448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26095188, "raw_average_key_size": 28, "raw_value_size": 108122606, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931971, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396356, "oldest_key_time": 0, "file_creation_time": 1732396356, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293106, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:40.335247 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293107.sst) /dev/shm/ToplingTemp/Topling-277024-jGk1Hs 2024/11/24-05:12:41.486807 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293107.sst): first pass time = 1.15's, 116.559'MB/sec 2024/11/24-05:12:42.809751 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293107.sst): old prealloc_size = 147639500, real_size = 31590592 2024/11/24-05:12:42.810348 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293107.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919944 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0296 GB avg-val = 117.90 avg-zval = 32.19 2024/11/24-05:12:42.812432 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293107.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:43.628956 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293107.sst): second pass time = 1.32's, 82.095'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.544'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3048.070'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 278.879'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35332.486'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.095'MB/sec zip pipeline throughput = 82.095'MB/sec entries = 919944 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.90 avg-zval = 32.19 usrkeys = 919944 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.90 avg-zval = 32.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919944 bytes = 3219824 } seq expand size = 6439608 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5132 value = 3.6629 dict = 1.00 all = 3.7627 } Zip/UnZip{ index = 0.0951 value = 0.2730 dict = 1.00 all = 0.2658 } ---------------------------- total value len = 2252.243562 GB avg = 0.200 KB (by entry num) total key len = 307.090634 GB avg = 0.027 KB total ukey len = 237.503064 GB avg = 0.015 KB total ukey num = 15.746078017 Billion total entry num = 11.287391965 Billion write speed all = 35.034697419 MB/sec (with seq num) write speed all = 33.798593552 MB/sec (without seq num) 2024/11/24-05:12:43.673075 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293107: 919944 keys, 35670758 bytes, temperature: kWarm 2024/11/24-05:12:43.673143 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396363673109, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293107, "file_size": 35670758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4454499314, "table_properties": {"data_size": 29610352, "index_size": 1750080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25758432, "raw_average_key_size": 28, "raw_value_size": 108459384, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919944, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396360, "oldest_key_time": 0, "file_creation_time": 1732396360, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293107, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:43.714186 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293108.sst) /dev/shm/ToplingTemp/Topling-277024-CcX6Js 2024/11/24-05:12:44.699341 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293108.sst): first pass time = 0.99's, 136.246'MB/sec 2024/11/24-05:12:46.017821 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293108.sst): old prealloc_size = 147639500, real_size = 30755952 2024/11/24-05:12:46.018420 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293108.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925903 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.96 avg-zval = 31.05 2024/11/24-05:12:46.019193 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293108.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:46.840811 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293108.sst): second pass time = 1.32's, 82.208'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.206'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3073.407'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 271.904'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35810.537'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.208'MB/sec zip pipeline throughput = 82.208'MB/sec entries = 925903 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.96 avg-zval = 31.05 usrkeys = 925903 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.96 avg-zval = 31.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925903 bytes = 3124944 } seq expand size = 6481321 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4453 value = 3.7665 dict = 1.00 all = 3.8641 } Zip/UnZip{ index = 0.0957 value = 0.2655 dict = 1.00 all = 0.2588 } ---------------------------- total value len = 2252.358336 GB avg = 0.200 KB (by entry num) total key len = 307.116560 GB avg = 0.027 KB total ukey len = 237.521582 GB avg = 0.015 KB total ukey num = 15.747003920 Billion total entry num = 11.288317868 Billion write speed all = 35.035083061 MB/sec (with seq num) write speed all = 33.798932147 MB/sec (without seq num) 2024/11/24-05:12:46.885474 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293108: 925903 keys, 34734218 bytes, temperature: kWarm 2024/11/24-05:12:46.885543 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396366885509, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293108, "file_size": 34734218, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4370794287, "largest_seqno": 4454499173, "table_properties": {"data_size": 28751456, "index_size": 1772864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25925284, "raw_average_key_size": 28, "raw_value_size": 108292542, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925903, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396363, "oldest_key_time": 0, "file_creation_time": 1732396363, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293108, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:46.921558 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293109.sst) /dev/shm/ToplingTemp/Topling-277024-bawrNt 2024/11/24-05:12:48.025129 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293109.sst): first pass time = 1.10's, 121.625'MB/sec 2024/11/24-05:12:49.389997 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293109.sst): old prealloc_size = 147639500, real_size = 32889248 2024/11/24-05:12:49.390570 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293109.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924988 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.10 avg-zval = 33.40 2024/11/24-05:12:49.391345 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293109.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:50.242491 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293109.sst): second pass time = 1.36's, 79.414'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.381'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3164.603'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 290.547'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30691.451'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.414'MB/sec zip pipeline throughput = 79.414'MB/sec entries = 924988 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 33.40 usrkeys = 924988 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 33.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924988 bytes = 3237472 } seq expand size = 6474916 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5128 value = 3.5057 dict = 1.00 all = 3.6293 } Zip/UnZip{ index = 0.0951 value = 0.2853 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 2252.473129 GB avg = 0.200 KB (by entry num) total key len = 307.142459 GB avg = 0.027 KB total ukey len = 237.540082 GB avg = 0.015 KB total ukey num = 15.747928908 Billion total entry num = 11.289242856 Billion write speed all = 35.035377546 MB/sec (with seq num) write speed all = 33.799182900 MB/sec (without seq num) 2024/11/24-05:12:50.290318 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293109: 924988 keys, 36981910 bytes, temperature: kWarm 2024/11/24-05:12:50.290384 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396370290352, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293109, "file_size": 36981910, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4454498858, "table_properties": {"data_size": 30898064, "index_size": 1759744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899664, "raw_average_key_size": 28, "raw_value_size": 108318072, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924988, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396366, "oldest_key_time": 0, "file_creation_time": 1732396366, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293109, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:50.327596 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293110.sst) /dev/shm/ToplingTemp/Topling-277024-6BYjSt 2024/11/24-05:12:51.426945 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293110.sst): first pass time = 1.10's, 122.093'MB/sec 2024/11/24-05:12:52.746517 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293110.sst): old prealloc_size = 147639500, real_size = 31296128 2024/11/24-05:12:52.747114 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293110.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920854 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.75 avg-zval = 31.82 2024/11/24-05:12:52.747871 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293110.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:53.575757 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293110.sst): second pass time = 1.32's, 82.218'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.525'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3037.752'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 272.317'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 30303.284'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.218'MB/sec zip pipeline throughput = 82.218'MB/sec entries = 920854 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.75 avg-zval = 31.82 usrkeys = 920854 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.75 avg-zval = 31.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920854 bytes = 3107904 } seq expand size = 6445978 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4464 value = 3.7005 dict = 1.00 all = 3.8063 } Zip/UnZip{ index = 0.0957 value = 0.2702 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 2252.588009 GB avg = 0.200 KB (by entry num) total key len = 307.168243 GB avg = 0.027 KB total ukey len = 237.558499 GB avg = 0.015 KB total ukey num = 15.748849762 Billion total entry num = 11.290163710 Billion write speed all = 35.035704415 MB/sec (with seq num) write speed all = 33.799465337 MB/sec (without seq num) 2024/11/24-05:12:53.618334 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293110: 920854 keys, 35261908 bytes, temperature: kWarm 2024/11/24-05:12:53.618400 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396373618367, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293110, "file_size": 35261908, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4349188220, "largest_seqno": 4454499191, "table_properties": {"data_size": 29302704, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25783912, "raw_average_key_size": 28, "raw_value_size": 108433868, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920854, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396370, "oldest_key_time": 0, "file_creation_time": 1732396370, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293110, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:53.658026 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293111.sst) /dev/shm/ToplingTemp/Topling-277024-WjkuIq 2024/11/24-05:12:54.723747 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293111.sst): first pass time = 1.07's, 125.945'MB/sec 2024/11/24-05:12:56.072782 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293111.sst): old prealloc_size = 147639500, real_size = 32794672 2024/11/24-05:12:56.073374 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293111.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930176 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.29 avg-zval = 33.10 2024/11/24-05:12:56.074177 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293111.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:12:56.934476 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293111.sst): second pass time = 1.35's, 80.232'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.827'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3138.622'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.12's, 295.337'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35216.287'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.232'MB/sec zip pipeline throughput = 80.232'MB/sec entries = 930176 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.29 avg-zval = 33.10 usrkeys = 930176 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.29 avg-zval = 33.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930176 bytes = 3255632 } seq expand size = 6511232 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5132 value = 3.5130 dict = 1.00 all = 3.6371 } Zip/UnZip{ index = 0.0951 value = 0.2847 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 2252.702693 GB avg = 0.200 KB (by entry num) total key len = 307.194288 GB avg = 0.027 KB total ukey len = 237.577103 GB avg = 0.015 KB total ukey num = 15.749779938 Billion total entry num = 11.291093886 Billion write speed all = 35.036019944 MB/sec (with seq num) write speed all = 33.799735848 MB/sec (without seq num) 2024/11/24-05:12:56.979715 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293111: 930176 keys, 36902166 bytes, temperature: kWarm 2024/11/24-05:12:56.979782 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396376979747, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293111, "file_size": 36902166, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4454498068, "table_properties": {"data_size": 30792416, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26044928, "raw_average_key_size": 28, "raw_value_size": 108172840, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930176, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396373, "oldest_key_time": 0, "file_creation_time": 1732396373, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293111, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:12:57.019169 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293112.sst) /dev/shm/ToplingTemp/Topling-277024-LjO2Xs 2024/11/24-05:12:58.061060 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293112.sst): first pass time = 1.04's, 128.827'MB/sec 2024/11/24-05:12:59.358748 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293112.sst): old prealloc_size = 147639500, real_size = 31364528 2024/11/24-05:12:59.359386 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293112.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.95 2024/11/24-05:12:59.361322 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293112.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:13:00.175149 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293112.sst): second pass time = 1.30's, 83.668'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.272'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2859.491'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 278.260'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34749.231'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.668'MB/sec zip pipeline throughput = 83.668'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.95 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4447 value = 3.6930 dict = 1.00 all = 3.7994 } Zip/UnZip{ index = 0.0957 value = 0.2708 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 2252.817605 GB avg = 0.200 KB (by entry num) total key len = 307.220029 GB avg = 0.027 KB total ukey len = 237.595489 GB avg = 0.015 KB total ukey num = 15.750699238 Billion total entry num = 11.292013186 Billion write speed all = 35.036391002 MB/sec (with seq num) write speed all = 33.800061086 MB/sec (without seq num) 2024/11/24-05:13:00.218382 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293112: 919300 keys, 35326386 bytes, temperature: kWarm 2024/11/24-05:13:00.218451 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396380218415, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293112, "file_size": 35326386, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4327320862, "largest_seqno": 4438700543, "table_properties": {"data_size": 29374176, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396377, "oldest_key_time": 0, "file_creation_time": 1732396377, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293112, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:13:00.255851 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293113.sst) /dev/shm/ToplingTemp/Topling-277024-Zud6Bt 2024/11/24-05:13:01.264528 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293113.sst): first pass time = 1.01's, 133.068'MB/sec 2024/11/24-05:13:02.609702 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293113.sst): old prealloc_size = 147639500, real_size = 32698880 2024/11/24-05:13:02.610321 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293113.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931670 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.06 avg-zval = 32.94 2024/11/24-05:13:02.611709 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293113.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:13:03.464833 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293113.sst): second pass time = 1.34's, 80.434'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.048'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2970.132'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 291.057'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31376.466'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.434'MB/sec zip pipeline throughput = 80.434'MB/sec entries = 931670 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 32.94 usrkeys = 931670 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 32.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931670 bytes = 3260864 } seq expand size = 6521690 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5126 value = 3.5230 dict = 1.00 all = 3.6462 } Zip/UnZip{ index = 0.0951 value = 0.2839 dict = 1.00 all = 0.2743 } ---------------------------- total value len = 2252.932258 GB avg = 0.199 KB (by entry num) total key len = 307.246115 GB avg = 0.027 KB total ukey len = 237.614122 GB avg = 0.015 KB total ukey num = 15.751630908 Billion total entry num = 11.292944856 Billion write speed all = 35.036739716 MB/sec (with seq num) write speed all = 33.800363458 MB/sec (without seq num) 2024/11/24-05:13:03.513018 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293113: 931670 keys, 36810752 bytes, temperature: kWarm 2024/11/24-05:13:03.513090 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396383513054, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293113, "file_size": 36810752, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4454499207, "table_properties": {"data_size": 30693296, "index_size": 1772480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26086760, "raw_average_key_size": 28, "raw_value_size": 108131064, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931670, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396380, "oldest_key_time": 0, "file_creation_time": 1732396380, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293113, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:13:03.554283 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293114.sst) /dev/shm/ToplingTemp/Topling-277024-aIi51t 2024/11/24-05:13:04.544146 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293114.sst): first pass time = 0.99's, 135.598'MB/sec 2024/11/24-05:13:05.848866 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293114.sst): old prealloc_size = 147639500, real_size = 31517760 2024/11/24-05:13:05.849428 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293114.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.12 2024/11/24-05:13:05.851374 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293114.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:13:06.664538 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293114.sst): second pass time = 1.30's, 83.194'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.499'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3267.811'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.12's, 284.637'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34697.004'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.194'MB/sec zip pipeline throughput = 83.194'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.12 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6738 dict = 1.00 all = 3.7830 } Zip/UnZip{ index = 0.0957 value = 0.2722 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2253.047170 GB avg = 0.199 KB (by entry num) total key len = 307.271856 GB avg = 0.027 KB total ukey len = 237.632508 GB avg = 0.015 KB total ukey num = 15.752550208 Billion total entry num = 11.293864156 Billion write speed all = 35.037130354 MB/sec (with seq num) write speed all = 33.800707591 MB/sec (without seq num) 2024/11/24-05:13:06.711775 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293114: 919300 keys, 35479602 bytes, temperature: kWarm 2024/11/24-05:13:06.711847 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396386711811, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293114, "file_size": 35479602, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4306075573, "largest_seqno": 4416505339, "table_properties": {"data_size": 29527600, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396383, "oldest_key_time": 0, "file_creation_time": 1732396383, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293114, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:13:06.748237 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293115.sst) /dev/shm/ToplingTemp/Topling-277024-WjDWEr 2024/11/24-05:13:07.660215 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293115.sst): first pass time = 0.91's, 147.180'MB/sec 2024/11/24-05:13:08.991169 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293115.sst): old prealloc_size = 147639500, real_size = 31698064 2024/11/24-05:13:08.991687 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293115.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931846 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0297 GB avg-val = 116.03 avg-zval = 31.86 2024/11/24-05:13:08.993692 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293115.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:13:09.827076 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293115.sst): second pass time = 1.33's, 81.317'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.841'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3574.112'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 285.328'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33103.209'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.317'MB/sec zip pipeline throughput = 81.317'MB/sec entries = 931846 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 31.86 usrkeys = 931846 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 31.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931846 bytes = 3261472 } seq expand size = 6522922 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5154 value = 3.6415 dict = 1.00 all = 3.7480 } Zip/UnZip{ index = 0.0951 value = 0.2746 dict = 1.00 all = 0.2668 } ---------------------------- total value len = 2253.161819 GB avg = 0.199 KB (by entry num) total key len = 307.297947 GB avg = 0.027 KB total ukey len = 237.651145 GB avg = 0.015 KB total ukey num = 15.753482054 Billion total entry num = 11.294796002 Billion write speed all = 35.037539980 MB/sec (with seq num) write speed all = 33.801068713 MB/sec (without seq num) 2024/11/24-05:13:09.874563 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293115: 931846 keys, 35810374 bytes, temperature: kWarm 2024/11/24-05:13:09.874631 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396389874596, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293115, "file_size": 35810374, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4454499227, "table_properties": {"data_size": 29692544, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26091688, "raw_average_key_size": 28, "raw_value_size": 108126156, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931846, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396386, "oldest_key_time": 0, "file_creation_time": 1732396386, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293115, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:13:09.916105 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293116.sst) /dev/shm/ToplingTemp/Topling-277024-9XwfXr 2024/11/24-05:13:10.915534 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293116.sst): first pass time = 1.00's, 134.299'MB/sec 2024/11/24-05:13:12.208943 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293116.sst): old prealloc_size = 147639500, real_size = 31406816 2024/11/24-05:13:12.209533 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293116.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.00 2024/11/24-05:13:12.211489 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293116.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:13:13.017815 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293116.sst): second pass time = 1.29's, 83.925'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.727'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3215.019'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 281.568'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32893.046'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.925'MB/sec zip pipeline throughput = 83.925'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 32.00 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 32.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4436 value = 3.6877 dict = 1.00 all = 3.7948 } Zip/UnZip{ index = 0.0958 value = 0.2712 dict = 1.00 all = 0.2635 } ---------------------------- total value len = 2253.276732 GB avg = 0.199 KB (by entry num) total key len = 307.323688 GB avg = 0.027 KB total ukey len = 237.669531 GB avg = 0.015 KB total ukey num = 15.754401354 Billion total entry num = 11.295715302 Billion write speed all = 35.037934849 MB/sec (with seq num) write speed all = 33.801416933 MB/sec (without seq num) 2024/11/24-05:13:13.064885 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293116: 919300 keys, 35368626 bytes, temperature: kWarm 2024/11/24-05:13:13.064957 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396393064920, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293116, "file_size": 35368626, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4283906683, "largest_seqno": 4393366154, "table_properties": {"data_size": 29416272, "index_size": 1760512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396389, "oldest_key_time": 0, "file_creation_time": 1732396389, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293116, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:13:13.106822 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293117.sst) /dev/shm/ToplingTemp/Topling-277024-w8Biir 2024/11/24-05:13:14.111299 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293117.sst): first pass time = 1.00's, 133.625'MB/sec 2024/11/24-05:13:15.436775 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293117.sst): old prealloc_size = 147639500, real_size = 30128816 2024/11/24-05:13:15.437389 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293117.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931162 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0281 GB avg-val = 116.14 avg-zval = 30.19 2024/11/24-05:13:15.438129 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293117.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:13:16.256087 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293117.sst): second pass time = 1.32's, 81.636'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.505'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3024.927'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 269.953'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32364.144'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.636'MB/sec zip pipeline throughput = 81.636'MB/sec entries = 931162 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.14 avg-zval = 30.19 usrkeys = 931162 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.14 avg-zval = 30.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931162 bytes = 3026288 } seq expand size = 6518134 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0281 GB dict = 1.08 MB all = 0.0340 GB } UnZip/Zip{ index = 10.4447 value = 3.8468 dict = 1.00 all = 3.9470 } Zip/UnZip{ index = 0.0957 value = 0.2600 dict = 1.00 all = 0.2534 } ---------------------------- total value len = 2253.391395 GB avg = 0.199 KB (by entry num) total key len = 307.349760 GB avg = 0.027 KB total ukey len = 237.688154 GB avg = 0.015 KB total ukey num = 15.755332516 Billion total entry num = 11.296646464 Billion write speed all = 35.038308029 MB/sec (with seq num) write speed all = 33.801742974 MB/sec (without seq num) 2024/11/24-05:13:16.300058 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293117: 931162 keys, 34004726 bytes, temperature: kWarm 2024/11/24-05:13:16.300129 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396396300092, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293117, "file_size": 34004726, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4393366155, "largest_seqno": 4454499237, "table_properties": {"data_size": 28112800, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26072536, "raw_average_key_size": 28, "raw_value_size": 108145264, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931162, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396393, "oldest_key_time": 0, "file_creation_time": 1732396393, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293117, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:13:16.338755 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293118.sst) /dev/shm/ToplingTemp/Topling-277024-EZPrcu 2024/11/24-05:13:17.202804 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293118.sst): first pass time = 0.86's, 155.342'MB/sec 2024/11/24-05:13:18.475795 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293118.sst): old prealloc_size = 147639500, real_size = 31612256 2024/11/24-05:13:18.476360 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293118.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919992 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0296 GB avg-val = 117.89 avg-zval = 32.21 2024/11/24-05:13:18.478353 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293118.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:13:19.289008 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293118.sst): second pass time = 1.27's, 85.253'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.893'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3234.027'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 283.336'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33167.508'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.253'MB/sec zip pipeline throughput = 85.253'MB/sec entries = 919992 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.89 avg-zval = 32.21 usrkeys = 919992 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.89 avg-zval = 32.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919992 bytes = 3219984 } seq expand size = 6439944 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5141 value = 3.6602 dict = 1.00 all = 3.7604 } Zip/UnZip{ index = 0.0951 value = 0.2732 dict = 1.00 all = 0.2659 } ---------------------------- total value len = 2253.506293 GB avg = 0.199 KB (by entry num) total key len = 307.375520 GB avg = 0.027 KB total ukey len = 237.706554 GB avg = 0.015 KB total ukey num = 15.756252508 Billion total entry num = 11.297566456 Billion write speed all = 35.038778564 MB/sec (with seq num) write speed all = 33.802164122 MB/sec (without seq num) 2024/11/24-05:13:19.334874 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293118: 919992 keys, 35692494 bytes, temperature: kWarm 2024/11/24-05:13:19.334946 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396399334909, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293118, "file_size": 35692494, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679976, "largest_seqno": 4454498321, "table_properties": {"data_size": 29632080, "index_size": 1750016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25759776, "raw_average_key_size": 28, "raw_value_size": 108458080, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919992, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396396, "oldest_key_time": 0, "file_creation_time": 1732396396, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293118, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:13:19.377166 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293119.sst) /dev/shm/ToplingTemp/Topling-277024-7RWHot 2024/11/24-05:13:20.361203 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293119.sst): first pass time = 0.98's, 136.400'MB/sec 2024/11/24-05:13:21.652856 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293119.sst): old prealloc_size = 147639500, real_size = 30789584 2024/11/24-05:13:21.653435 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293119.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925853 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.97 avg-zval = 31.09 2024/11/24-05:13:21.654195 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293119.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:13:22.466403 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293119.sst): second pass time = 1.29's, 83.891'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.737'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3291.009'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 274.353'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33871.326'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.891'MB/sec zip pipeline throughput = 83.891'MB/sec entries = 925853 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.97 avg-zval = 31.09 usrkeys = 925853 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.97 avg-zval = 31.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925853 bytes = 3124768 } seq expand size = 6480971 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4432 value = 3.7622 dict = 1.00 all = 3.8604 } Zip/UnZip{ index = 0.0958 value = 0.2658 dict = 1.00 all = 0.2590 } ---------------------------- total value len = 2253.621068 GB avg = 0.199 KB (by entry num) total key len = 307.401444 GB avg = 0.027 KB total ukey len = 237.725071 GB avg = 0.015 KB total ukey num = 15.757178361 Billion total entry num = 11.298492309 Billion write speed all = 35.039180342 MB/sec (with seq num) write speed all = 33.802518320 MB/sec (without seq num) 2024/11/24-05:13:22.510938 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293119: 925853 keys, 34767698 bytes, temperature: kWarm 2024/11/24-05:13:22.511019 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396402510971, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293119, "file_size": 34767698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4370465821, "largest_seqno": 4454499251, "table_properties": {"data_size": 28784832, "index_size": 1773120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923884, "raw_average_key_size": 28, "raw_value_size": 108293910, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925853, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396399, "oldest_key_time": 0, "file_creation_time": 1732396399, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293119, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:13:22.558020 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293120.sst) /dev/shm/ToplingTemp/Topling-277024-cqB7Ut 2024/11/24-05:13:22.699580 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293120.sst): first pass time = 0.14's, 206.124'MB/sec 2024/11/24-05:13:22.987572 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293120.sst): old prealloc_size = 147639500, real_size = 7628864 2024/11/24-05:13:22.987752 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293120.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 205438 prefix = 4 raw-key = 0.0033 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0234 GB zip-val = 0.0072 GB avg-val = 114.00 avg-zval = 34.95 2024/11/24-05:13:22.987956 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293120.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.029 GB 2024/11/24-05:13:23.171065 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00/293120.sst): second pass time = 0.29's, 81.510'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 46.139'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2501.097'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.332'MB/sec write SST data time = 0.03's, 298.264'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.244'MB, throughput = 8818.386'MB/sec zip my value time = 0.29's, unzip length = 0.023'GB zip my value throughput = 81.510'MB/sec zip pipeline throughput = 81.510'MB/sec entries = 205438 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.00 avg-zval = 34.95 usrkeys = 205438 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.00 avg-zval = 34.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 205438 bytes = 642016 } seq expand size = 1438066 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0041 GB value = 0.0234 GB dict = 0.24 MB all = 0.0292 GB } __ZipSize{ index = 0.0004 GB value = 0.0072 GB dict = 0.24 MB all = 0.0085 GB } UnZip/Zip{ index = 10.3531 value = 3.2616 dict = 1.00 all = 3.4462 } Zip/UnZip{ index = 0.0966 value = 0.3066 dict = 1.00 all = 0.2902 } ---------------------------- total value len = 2253.645926 GB avg = 0.199 KB (by entry num) total key len = 307.407196 GB avg = 0.027 KB total ukey len = 237.729180 GB avg = 0.015 KB total ukey num = 15.757383799 Billion total entry num = 11.298697747 Billion write speed all = 35.039261330 MB/sec (with seq num) write speed all = 33.802588745 MB/sec (without seq num) 2024/11/24-05:13:23.188205 139957215163968 [compaction_job.cc:1948] [default] [JOB 42006] Generated table #293120: 205438 keys, 8464997 bytes, temperature: kWarm 2024/11/24-05:13:23.188275 139957215163968 EVENT_LOG_v1 {"time_micros": 1732396403188235, "cf_name": "default", "job": 42006, "event": "table_file_creation", "file_number": 293120, "file_size": 8464997, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4432162601, "largest_seqno": 4454311031, "table_properties": {"data_size": 7180464, "index_size": 396864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5752264, "raw_average_key_size": 28, "raw_value_size": 23419932, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 205438, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732396402, "oldest_key_time": 0, "file_creation_time": 1732396402, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293120, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:13:23.200442 139957215163968 [compaction_job.cc:703] job-42006: subcompact[0], size: 0.717541 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42006/att-00 [293100,293101,293102,293103,293104,293105,293106,293107,293108,293109,293110,293111,293112,293113,293114,293115,293116,293117,293118,293119,293120] 2024/11/24-05:13:23.356602 139957215163968 (Original Log Time 2024/11/24-05:12:13.705742) EVENT_LOG_v1 {"time_micros": 1732396333705696, "job": 42006, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [293097], "files_L3": [293058, 293059, 293060, 293061, 293062, 293063, 293064, 293065, 293066, 293067, 293068, 293069, 293070, 293015, 293016, 293017, 293018, 293019, 293020, 293021, 293022, 293023], "score": 0.972601, "input_data_size": 741933697, "oldest_snapshot_seqno": 4454509438} 2024/11/24-05:13:23.356609 139957215163968 (Original Log Time 2024/11/24-05:13:23.354369) [compaction_job.cc:2022] [default] [JOB 42006] Compacted 1@2 + 22@3 files to L3 => 717540815 bytes 2024/11/24-05:13:23.356611 139957215163968 (Original Log Time 2024/11/24-05:13:23.356529) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 10.7 rd, 10.3 wr, level 3, files in(1, 22) out(21 +0 blob) MB in(27.1, 680.5 +0.0 blob) out(684.3 +0.0 blob), read-write-amplify(51.3) write-amplify(25.2) OK, records in: 18825369, records dropped: 102733 output_compression: Snappy 2024/11/24-05:13:23.356614 139957215163968 (Original Log Time 2024/11/24-05:13:23.356575) EVENT_LOG_v1 {"time_micros": 1732396403356552, "cf": "default", "job": 42006, "event": "compaction_finished", "compaction_time_micros": 69494706, "compaction_time_cpu_micros": 29797221, "output_level": 3, "num_output_files": 21, "total_output_size": 717540815, "num_input_records": 18825369, "num_output_records": 18722636, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 496882002, "file_range_sync_nanos": 0, "file_fsync_nanos": 14302, "file_prepare_write_nanos": 50158, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/24-05:13:23.492763 139957215163968 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732396333,"executesMs":69787,"compactionJobId":42006,"attempt":0,"compactionInputRawBytes":2728116466,"compactionInputZipBytes":741933697,"compactionOutputRawBytes":0,"compactionOutputZipBytes":717540815,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c8aa7f7716d3beb74c3b8b77e99d81b2-8d1cc10a19c8445c-00"}}, response = 2024/11/24-05:13:23.492855 139957215163968 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42006/att-00: olev 3, work 69.788 s, result 5.983 ms, install 2.261 ms, input{raw 2.541 GiB zip 707.563 MiB}