2024/11/23-06:14:33.810395 140024953251392 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00: fork to child time = 0.009258 sec 2024/11/23-06:14:33.811206 140024953251392 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00 2024/11/23-06:14:33.818581 140024953251392 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00 2024/11/23-06:14:33.819709 140024953251392 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-06:14:33.819987 140024953251392 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-06:14:33.866144 140024953251392 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20846/att-00/MANIFEST-206347 2024/11/23-06:14:33.882068 140024953251392 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20846/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 248133, last_sequence is 3830092538, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 247965 2024/11/23-06:14:33.882074 140024953251392 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-06:14:33.882174 140024953251392 [version_set.cc:5721] Creating manifest 248133 2024/11/23-06:14:33.917239 140024953251392 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-06:14:33.939146 140024953251392 [compaction_job.cc:2437] [default] [JOB 20846] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-06:14:33.939165 140024953251392 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 15597 Base level 2, inputs: [248095(27MB)], [248047(34MB) 248048(36MB) 248049(36MB) 248050(36MB) 248051(35MB) 248052(35MB) 248053(35MB) 248054(35MB) 248055(11MB) 248013(34MB) 248014(36MB) 248015(35MB) 248016(33MB) 248017(35MB) 248127(35MB)] 2024/11/23-06:14:33.946001 140024953251392 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:14:33.946038 140024953251392 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:14:33.946057 140024953251392 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:14:33.946062 140024953251392 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:14:33.946073 140024953251392 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:14:34.403731 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248134.sst) /dev/shm/ToplingTemp/Topling-431566-Zuiu8b 2024/11/23-06:14:34.955386 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248134.sst): first pass time = 0.55's, 243.319'MB/sec 2024/11/23-06:14:36.650691 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248134.sst): old prealloc_size = 147639500, real_size = 35138192 2024/11/23-06:14:36.651369 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248134.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1083057 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0328 GB avg-val = 95.92 avg-zval = 30.27 2024/11/23-06:14:36.651590 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248134.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:14:37.463972 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248134.sst): second pass time = 1.69's, 61.314'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 44.667'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3162.957'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.404'MB/sec write SST data time = 0.14's, 255.613'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.033'MB, throughput = 32028.909'MB/sec zip my value time = 1.69's, unzip length = 0.104'GB zip my value throughput = 61.314'MB/sec zip pipeline throughput = 61.314'MB/sec entries = 1083057 avg-key = 28.00 avg-zkey = 1.93 avg-val = 95.92 avg-zval = 30.27 usrkeys = 1083057 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.46 avg-zval = 30.27 TagRS{ kinds = RS_Key0_TagN bytes = 169320 } TagArray{ size = 82018 bytes = 246064 } seq expand size = 574126 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0217 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0328 GB dict = 1.03 MB all = 0.0363 GB } UnZip/Zip{ index = 10.3792 value = 3.1693 dict = 1.00 all = 3.6948 } Zip/UnZip{ index = 0.0963 value = 0.3155 dict = 1.00 all = 0.2707 } ---------------------------- total value len = 0.104466 GB avg = 0.096 KB (by entry num) total key len = 0.030326 GB avg = 0.028 KB total ukey len = 0.021661 GB avg = 0.020 KB total ukey num = 0.001083057 Billion total entry num = 0.001083057 Billion write speed all = 0.001752037 MB/sec (with seq num) write speed all = 0.001639416 MB/sec (without seq num) 2024/11/23-06:14:37.517472 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248134: 1083057 keys, 36326355 bytes, temperature: kWarm 2024/11/23-06:14:37.517563 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313677517510, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248134, "file_size": 36326355, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3829906710, "table_properties": {"data_size": 32780288, "index_size": 2086976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30325596, "raw_average_key_size": 28, "raw_value_size": 103892147, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1083057, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313674, "oldest_key_time": 0, "file_creation_time": 1732313674, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248134, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:14:37.562236 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248135.sst) /dev/shm/ToplingTemp/Topling-431566-vg9mW9 2024/11/23-06:14:38.380616 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248135.sst): first pass time = 0.82's, 164.012'MB/sec 2024/11/23-06:14:40.069809 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248135.sst): old prealloc_size = 147639500, real_size = 35030464 2024/11/23-06:14:40.070323 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248135.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923978 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0330 GB avg-val = 117.26 avg-zval = 35.76 2024/11/23-06:14:40.070914 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248135.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:14:40.903492 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248135.sst): second pass time = 1.69's, 64.189'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 45.639'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3575.037'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 287.813'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34128.855'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.189'MB/sec zip pipeline throughput = 72.832'MB/sec entries = 923978 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.26 avg-zval = 35.76 usrkeys = 923978 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.33 avg-zval = 35.76 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 536614 bytes = 2012320 } seq expand size = 3756298 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5127 value = 3.2791 dict = 1.00 all = 3.5276 } Zip/UnZip{ index = 0.0951 value = 0.3050 dict = 1.00 all = 0.2835 } ---------------------------- total value len = 0.216569 GB avg = 0.108 KB (by entry num) total key len = 0.056197 GB avg = 0.028 KB total ukey len = 0.040141 GB avg = 0.020 KB total ukey num = 0.002007035 Billion total entry num = 0.002007035 Billion write speed all = 0.003545279 MB/sec (with seq num) write speed all = 0.003336587 MB/sec (without seq num) 2024/11/23-06:14:40.936330 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248135: 923978 keys, 38048066 bytes, temperature: kWarm 2024/11/23-06:14:40.936391 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313680936360, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248135, "file_size": 38048066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3829905074, "table_properties": {"data_size": 33041456, "index_size": 1757824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25871384, "raw_average_key_size": 28, "raw_value_size": 108346440, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923978, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313677, "oldest_key_time": 0, "file_creation_time": 1732313677, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248135, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:14:40.976936 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248136.sst) /dev/shm/ToplingTemp/Topling-431566-9yVzLd 2024/11/23-06:14:41.782629 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248136.sst): first pass time = 0.81's, 166.595'MB/sec 2024/11/23-06:14:43.327060 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248136.sst): old prealloc_size = 147639500, real_size = 33490976 2024/11/23-06:14:43.327596 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248136.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935743 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0315 GB avg-val = 115.43 avg-zval = 33.63 2024/11/23-06:14:43.328425 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248136.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:14:44.240309 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248136.sst): second pass time = 1.54's, 69.981'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.445'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3471.512'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 295.714'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 27876.918'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 69.981'MB/sec zip pipeline throughput = 125.241'MB/sec entries = 935743 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.43 avg-zval = 33.63 usrkeys = 935743 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.43 avg-zval = 33.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935743 bytes = 3509056 } seq expand size = 6550201 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4432 value = 3.4329 dict = 1.00 all = 3.5461 } Zip/UnZip{ index = 0.0958 value = 0.2913 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 0.331136 GB avg = 0.113 KB (by entry num) total key len = 0.082398 GB avg = 0.028 KB total ukey len = 0.058856 GB avg = 0.020 KB total ukey num = 0.002942778 Billion total entry num = 0.002942778 Billion write speed all = 0.005374679 MB/sec (with seq num) write speed all = 0.005068702 MB/sec (without seq num) 2024/11/23-06:14:44.272718 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248136: 935743 keys, 37849176 bytes, temperature: kWarm 2024/11/23-06:14:44.272775 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313684272744, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248136, "file_size": 37849176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3829906232, "table_properties": {"data_size": 31464784, "index_size": 1792064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26200804, "raw_average_key_size": 28, "raw_value_size": 108016934, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935743, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313680, "oldest_key_time": 0, "file_creation_time": 1732313680, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248136, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:14:44.309691 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248137.sst) /dev/shm/ToplingTemp/Topling-431566-tQhSCa 2024/11/23-06:14:45.113293 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248137.sst): first pass time = 0.80's, 167.027'MB/sec 2024/11/23-06:14:46.671775 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248137.sst): old prealloc_size = 147639500, real_size = 33777552 2024/11/23-06:14:46.672322 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248137.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924613 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0318 GB avg-val = 117.16 avg-zval = 34.37 2024/11/23-06:14:46.674434 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248137.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:14:47.612889 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248137.sst): second pass time = 1.56's, 69.542'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.486'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3402.676'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.13's, 296.268'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26177.769'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.542'MB/sec zip pipeline throughput = 129.793'MB/sec entries = 924613 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.16 avg-zval = 34.37 usrkeys = 924613 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.16 avg-zval = 34.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924613 bytes = 3467312 } seq expand size = 6472291 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4435 value = 3.4092 dict = 1.00 all = 3.5227 } Zip/UnZip{ index = 0.0958 value = 0.2933 dict = 1.00 all = 0.2839 } ---------------------------- total value len = 0.445937 GB avg = 0.115 KB (by entry num) total key len = 0.108287 GB avg = 0.028 KB total ukey len = 0.077348 GB avg = 0.020 KB total ukey num = 0.003867391 Billion total entry num = 0.003867391 Billion write speed all = 0.007202905 MB/sec (with seq num) write speed all = 0.006800808 MB/sec (without seq num) 2024/11/23-06:14:47.650645 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248137: 924613 keys, 38100992 bytes, temperature: kWarm 2024/11/23-06:14:47.650701 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313687650672, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248137, "file_size": 38100992, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3277348698, "largest_seqno": 3829905125, "table_properties": {"data_size": 31775552, "index_size": 1770688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25889164, "raw_average_key_size": 28, "raw_value_size": 108328622, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924613, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313684, "oldest_key_time": 0, "file_creation_time": 1732313684, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248137, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:14:47.689137 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248138.sst) /dev/shm/ToplingTemp/Topling-431566-38uc5b 2024/11/23-06:14:48.475908 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248138.sst): first pass time = 0.79's, 170.600'MB/sec 2024/11/23-06:14:50.057213 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248138.sst): old prealloc_size = 147639500, real_size = 32980640 2024/11/23-06:14:50.057756 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248138.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928374 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.57 avg-zval = 33.37 2024/11/23-06:14:50.058543 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248138.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:14:50.993642 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248138.sst): second pass time = 1.58's, 68.469'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.726'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3367.368'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 275.593'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26638.589'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.469'MB/sec zip pipeline throughput = 134.224'MB/sec entries = 928374 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.57 avg-zval = 33.37 usrkeys = 928374 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.57 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928374 bytes = 3365376 } seq expand size = 6498618 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5130 value = 3.4931 dict = 1.00 all = 3.6081 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 0.560659 GB avg = 0.117 KB (by entry num) total key len = 0.134281 GB avg = 0.028 KB total ukey len = 0.095915 GB avg = 0.020 KB total ukey num = 0.004795765 Billion total entry num = 0.004795765 Billion write speed all = 0.009031310 MB/sec (with seq num) write speed all = 0.008532712 MB/sec (without seq num) 2024/11/23-06:14:51.024954 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248138: 928374 keys, 37199422 bytes, temperature: kWarm 2024/11/23-06:14:51.025026 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313691024985, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248138, "file_size": 37199422, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3829905405, "table_properties": {"data_size": 30982224, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25994472, "raw_average_key_size": 28, "raw_value_size": 108223256, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928374, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313687, "oldest_key_time": 0, "file_creation_time": 1732313687, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248138, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:14:51.073538 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248139.sst) /dev/shm/ToplingTemp/Topling-431566-xLDTRc 2024/11/23-06:14:51.898568 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248139.sst): first pass time = 0.82's, 162.690'MB/sec 2024/11/23-06:14:53.407993 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248139.sst): old prealloc_size = 147639500, real_size = 33480944 2024/11/23-06:14:53.408525 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248139.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925879 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.96 avg-zval = 34.01 2024/11/23-06:14:53.409334 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248139.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:14:54.351399 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248139.sst): second pass time = 1.51's, 71.776'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.954'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3464.262'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.13's, 292.304'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28645.693'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.776'MB/sec zip pipeline throughput = 116.052'MB/sec entries = 925879 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 34.01 usrkeys = 925879 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 34.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925879 bytes = 3356320 } seq expand size = 6481153 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5114 value = 3.4392 dict = 1.00 all = 3.5609 } Zip/UnZip{ index = 0.0951 value = 0.2908 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 0.675433 GB avg = 0.118 KB (by entry num) total key len = 0.160206 GB avg = 0.028 KB total ukey len = 0.114433 GB avg = 0.020 KB total ukey num = 0.005721644 Billion total entry num = 0.005721644 Billion write speed all = 0.010859333 MB/sec (with seq num) write speed all = 0.010264500 MB/sec (without seq num) 2024/11/23-06:14:54.385958 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248139: 925879 keys, 37692466 bytes, temperature: kWarm 2024/11/23-06:14:54.386022 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313694385988, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248139, "file_size": 37692466, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3829906311, "table_properties": {"data_size": 31487648, "index_size": 1761664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924612, "raw_average_key_size": 28, "raw_value_size": 108293166, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925879, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313691, "oldest_key_time": 0, "file_creation_time": 1732313691, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248139, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:14:54.424976 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248140.sst) /dev/shm/ToplingTemp/Topling-431566-N6rfxb 2024/11/23-06:14:55.213791 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248140.sst): first pass time = 0.79's, 170.159'MB/sec 2024/11/23-06:14:56.738473 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248140.sst): old prealloc_size = 147639500, real_size = 33268848 2024/11/23-06:14:56.739072 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248140.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928210 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.60 avg-zval = 33.69 2024/11/23-06:14:56.739851 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248140.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:14:57.694659 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248140.sst): second pass time = 1.52's, 71.039'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.084'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3059.537'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 293.970'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33618.659'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.039'MB/sec zip pipeline throughput = 111.884'MB/sec entries = 928210 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.60 avg-zval = 33.69 usrkeys = 928210 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.60 avg-zval = 33.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928210 bytes = 3248752 } seq expand size = 6497470 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5131 value = 3.4610 dict = 1.00 all = 3.5915 } Zip/UnZip{ index = 0.0951 value = 0.2889 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 0.790159 GB avg = 0.119 KB (by entry num) total key len = 0.186196 GB avg = 0.028 KB total ukey len = 0.132997 GB avg = 0.020 KB total ukey num = 0.006649854 Billion total entry num = 0.006649854 Billion write speed all = 0.012687410 MB/sec (with seq num) write speed all = 0.011996109 MB/sec (without seq num) 2024/11/23-06:14:57.733415 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248140: 928210 keys, 37371004 bytes, temperature: kWarm 2024/11/23-06:14:57.733479 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313697733447, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248140, "file_size": 37371004, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3829906174, "table_properties": {"data_size": 31270816, "index_size": 1765824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989880, "raw_average_key_size": 28, "raw_value_size": 108227928, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928210, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313694, "oldest_key_time": 0, "file_creation_time": 1732313694, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248140, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:14:57.768587 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248141.sst) /dev/shm/ToplingTemp/Topling-431566-TKmoqc 2024/11/23-06:14:58.604347 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248141.sst): first pass time = 0.84's, 160.601'MB/sec 2024/11/23-06:15:00.109066 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248141.sst): old prealloc_size = 147639500, real_size = 33213776 2024/11/23-06:15:00.109658 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248141.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928414 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.57 avg-zval = 33.61 2024/11/23-06:15:00.110450 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248141.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:15:01.069116 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248141.sst): second pass time = 1.50's, 71.955'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.991'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3118.044'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.13's, 293.127'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35981.533'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 71.955'MB/sec zip pipeline throughput = 81.113'MB/sec entries = 928414 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.57 avg-zval = 33.61 usrkeys = 928414 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.57 avg-zval = 33.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928414 bytes = 3365520 } seq expand size = 6498898 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4449 value = 3.4682 dict = 1.00 all = 3.5856 } Zip/UnZip{ index = 0.0957 value = 0.2883 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 0.904880 GB avg = 0.119 KB (by entry num) total key len = 0.212192 GB avg = 0.028 KB total ukey len = 0.151565 GB avg = 0.020 KB total ukey num = 0.007578268 Billion total entry num = 0.007578268 Billion write speed all = 0.014515342 MB/sec (with seq num) write speed all = 0.013727560 MB/sec (without seq num) 2024/11/23-06:15:01.103662 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248141: 928414 keys, 37432450 bytes, temperature: kWarm 2024/11/23-06:15:01.103718 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313701103689, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248141, "file_size": 37432450, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3829906430, "table_properties": {"data_size": 31203776, "index_size": 1777728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995592, "raw_average_key_size": 28, "raw_value_size": 108222256, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928414, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313697, "oldest_key_time": 0, "file_creation_time": 1732313697, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248141, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:15:01.140510 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248142.sst) /dev/shm/ToplingTemp/Topling-431566-w37MCa 2024/11/23-06:15:02.132621 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248142.sst): first pass time = 0.99's, 135.290'MB/sec 2024/11/23-06:15:03.790261 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248142.sst): old prealloc_size = 147639500, real_size = 33150032 2024/11/23-06:15:03.790842 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248142.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928473 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.56 avg-zval = 33.55 2024/11/23-06:15:03.793356 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248142.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:15:04.745369 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248142.sst): second pass time = 1.66's, 65.309'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.127'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3114.652'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 284.179'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34828.834'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.309'MB/sec zip pipeline throughput = 130.645'MB/sec entries = 928473 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 33.55 usrkeys = 928473 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928473 bytes = 3365728 } seq expand size = 6499311 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5126 value = 3.4740 dict = 1.00 all = 3.5917 } Zip/UnZip{ index = 0.0951 value = 0.2879 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 1.019600 GB avg = 0.120 KB (by entry num) total key len = 0.238189 GB avg = 0.028 KB total ukey len = 0.170135 GB avg = 0.020 KB total ukey num = 0.008506741 Billion total entry num = 0.008506741 Billion write speed all = 0.016343055 MB/sec (with seq num) write speed all = 0.015458797 MB/sec (without seq num) 2024/11/23-06:15:04.778130 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248142: 928473 keys, 37369356 bytes, temperature: kWarm 2024/11/23-06:15:04.778193 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313704778162, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248142, "file_size": 37369356, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3829905346, "table_properties": {"data_size": 31151360, "index_size": 1766400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997244, "raw_average_key_size": 28, "raw_value_size": 108220578, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928473, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313701, "oldest_key_time": 0, "file_creation_time": 1732313701, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248142, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:15:04.818645 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248143.sst) /dev/shm/ToplingTemp/Topling-431566-eLh0Za 2024/11/23-06:15:05.721052 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248143.sst): first pass time = 0.90's, 148.740'MB/sec 2024/11/23-06:15:07.192176 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248143.sst): old prealloc_size = 147639500, real_size = 33267584 2024/11/23-06:15:07.192727 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248143.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930300 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.27 avg-zval = 33.60 2024/11/23-06:15:07.193460 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248143.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:15:08.084195 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248143.sst): second pass time = 1.47's, 73.609'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.559'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3366.988'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.14's, 268.261'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33331.912'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.609'MB/sec zip pipeline throughput = 73.609'MB/sec entries = 930300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.27 avg-zval = 33.60 usrkeys = 930300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.90 avg-zval = 33.60 TagRS{ kinds = RS_Key0_TagN bytes = 145440 } TagArray{ size = 747303 bytes = 2802400 } seq expand size = 5231121 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4470 value = 3.4610 dict = 1.00 all = 3.6206 } Zip/UnZip{ index = 0.0957 value = 0.2889 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 1.133000 GB avg = 0.120 KB (by entry num) total key len = 0.264237 GB avg = 0.028 KB total ukey len = 0.188741 GB avg = 0.020 KB total ukey num = 0.009437041 Billion total entry num = 0.009437041 Billion write speed all = 0.018154195 MB/sec (with seq num) write speed all = 0.017173278 MB/sec (without seq num) 2024/11/23-06:15:08.117135 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248143: 930300 keys, 37070962 bytes, temperature: kWarm 2024/11/23-06:15:08.117200 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313708117167, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248143, "file_size": 37070962, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3829906384, "table_properties": {"data_size": 31253808, "index_size": 1780992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26048400, "raw_average_key_size": 28, "raw_value_size": 108169468, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313704, "oldest_key_time": 0, "file_creation_time": 1732313704, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248143, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:15:08.156209 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248144.sst) /dev/shm/ToplingTemp/Topling-431566-BDvrKc 2024/11/23-06:15:09.025163 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248144.sst): first pass time = 0.87's, 154.465'MB/sec 2024/11/23-06:15:10.430790 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248144.sst): old prealloc_size = 147639500, real_size = 30576944 2024/11/23-06:15:10.431316 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248144.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928332 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.58 avg-zval = 30.77 2024/11/23-06:15:10.432225 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248144.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:15:11.298371 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248144.sst): second pass time = 1.40's, 77.030'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 40.239'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3516.407'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 270.237'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32255.773'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.030'MB/sec zip pipeline throughput = 77.030'MB/sec entries = 928332 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.58 avg-zval = 30.77 usrkeys = 928332 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.58 avg-zval = 30.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928332 bytes = 3481264 } seq expand size = 6498324 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4448 value = 3.7884 dict = 1.00 all = 3.8447 } Zip/UnZip{ index = 0.0957 value = 0.2640 dict = 1.00 all = 0.2601 } ---------------------------- total value len = 1.247723 GB avg = 0.120 KB (by entry num) total key len = 0.290230 GB avg = 0.028 KB total ukey len = 0.207307 GB avg = 0.020 KB total ukey num = 0.010365373 Billion total entry num = 0.010365373 Billion write speed all = 0.019981675 MB/sec (with seq num) write speed all = 0.018904308 MB/sec (without seq num) 2024/11/23-06:15:11.335261 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248144: 928332 keys, 34909858 bytes, temperature: kWarm 2024/11/23-06:15:11.335334 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313711335294, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248144, "file_size": 34909858, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3286287116, "largest_seqno": 3829907112, "table_properties": {"data_size": 28567072, "index_size": 1777600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25993296, "raw_average_key_size": 28, "raw_value_size": 108224524, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928332, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313708, "oldest_key_time": 0, "file_creation_time": 1732313708, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248144, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:15:11.373597 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248145.sst) /dev/shm/ToplingTemp/Topling-431566-LVaFsd 2024/11/23-06:15:12.135297 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248145.sst): first pass time = 0.76's, 176.216'MB/sec 2024/11/23-06:15:13.620677 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248145.sst): old prealloc_size = 147639500, real_size = 33499200 2024/11/23-06:15:13.621213 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248145.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926996 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.79 avg-zval = 33.99 2024/11/23-06:15:13.622643 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248145.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:15:14.573377 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248145.sst): second pass time = 1.48's, 72.916'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.587'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3423.521'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 290.234'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32163.724'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 72.916'MB/sec zip pipeline throughput = 81.526'MB/sec entries = 926996 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 33.99 usrkeys = 926996 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 33.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926996 bytes = 3360368 } seq expand size = 6488972 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5145 value = 3.4365 dict = 1.00 all = 3.5589 } Zip/UnZip{ index = 0.0951 value = 0.2910 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 1.362474 GB avg = 0.121 KB (by entry num) total key len = 0.316186 GB avg = 0.028 KB total ukey len = 0.225847 GB avg = 0.020 KB total ukey num = 0.011292369 Billion total entry num = 0.011292369 Billion write speed all = 0.021808862 MB/sec (with seq num) write speed all = 0.020635194 MB/sec (without seq num) 2024/11/23-06:15:14.606748 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248145: 926996 keys, 37713706 bytes, temperature: kWarm 2024/11/23-06:15:14.606819 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313714606787, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248145, "file_size": 37713706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3829907128, "table_properties": {"data_size": 31503984, "index_size": 1763264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955888, "raw_average_key_size": 28, "raw_value_size": 108261920, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926996, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313711, "oldest_key_time": 0, "file_creation_time": 1732313711, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248145, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:15:14.646922 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248146.sst) /dev/shm/ToplingTemp/Topling-431566-fjZ179 2024/11/23-06:15:15.491080 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248146.sst): first pass time = 0.84's, 159.003'MB/sec 2024/11/23-06:15:16.925564 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248146.sst): old prealloc_size = 147639500, real_size = 33081424 2024/11/23-06:15:16.926110 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248146.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929585 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.38 avg-zval = 33.41 2024/11/23-06:15:16.926897 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248146.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:15:17.878687 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248146.sst): second pass time = 1.43's, 75.479'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.018'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3403.946'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 290.066'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34160.845'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.479'MB/sec zip pipeline throughput = 75.479'MB/sec entries = 929585 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.38 avg-zval = 33.41 usrkeys = 929585 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.38 avg-zval = 33.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929585 bytes = 3253568 } seq expand size = 6507095 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.3752 value = 3.4836 dict = 1.00 all = 3.6092 } Zip/UnZip{ index = 0.0964 value = 0.2871 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 1.477171 GB avg = 0.121 KB (by entry num) total key len = 0.342215 GB avg = 0.028 KB total ukey len = 0.244439 GB avg = 0.020 KB total ukey num = 0.012221954 Billion total entry num = 0.012221954 Billion write speed all = 0.023636121 MB/sec (with seq num) write speed all = 0.022365892 MB/sec (without seq num) 2024/11/23-06:15:17.921174 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248146: 929585 keys, 37188188 bytes, temperature: kWarm 2024/11/23-06:15:17.921246 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313717921205, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248146, "file_size": 37188188, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3604582272, "largest_seqno": 3829907149, "table_properties": {"data_size": 31056896, "index_size": 1791936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26028380, "raw_average_key_size": 28, "raw_value_size": 108189466, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929585, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313714, "oldest_key_time": 0, "file_creation_time": 1732313714, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248146, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:15:17.957014 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248147.sst) /dev/shm/ToplingTemp/Topling-431566-gp8TRb 2024/11/23-06:15:18.829075 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248147.sst): first pass time = 0.87's, 153.915'MB/sec 2024/11/23-06:15:20.249912 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248147.sst): old prealloc_size = 147639500, real_size = 32309792 2024/11/23-06:15:20.250460 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248147.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933258 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 115.82 avg-zval = 32.47 2024/11/23-06:15:20.251268 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248147.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:15:21.214167 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248147.sst): second pass time = 1.42's, 76.110'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 37.899'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3374.971'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.285'MB/sec write SST data time = 0.14's, 252.365'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33750.102'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.110'MB/sec zip pipeline throughput = 76.110'MB/sec entries = 933258 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.82 avg-zval = 32.47 usrkeys = 933258 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.82 avg-zval = 32.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933258 bytes = 3383072 } seq expand size = 6532806 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5131 value = 3.5671 dict = 1.00 all = 3.6728 } Zip/UnZip{ index = 0.0951 value = 0.2803 dict = 1.00 all = 0.2723 } ---------------------------- total value len = 1.591790 GB avg = 0.121 KB (by entry num) total key len = 0.368346 GB avg = 0.028 KB total ukey len = 0.263104 GB avg = 0.020 KB total ukey num = 0.013155212 Billion total entry num = 0.013155212 Billion write speed all = 0.025463546 MB/sec (with seq num) write speed all = 0.024096382 MB/sec (without seq num) 2024/11/23-06:15:21.245275 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248147: 933258 keys, 36543436 bytes, temperature: kWarm 2024/11/23-06:15:21.245336 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313721245304, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248147, "file_size": 36543436, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3829907168, "table_properties": {"data_size": 30300880, "index_size": 1775424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26131224, "raw_average_key_size": 28, "raw_value_size": 108086508, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933258, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313717, "oldest_key_time": 0, "file_creation_time": 1732313717, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248147, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:15:21.280567 140024953251392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248148.sst) /dev/shm/ToplingTemp/Topling-431566-vISCIc 2024/11/23-06:15:21.549166 140024953251392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248148.sst): first pass time = 0.27's, 248.856'MB/sec 2024/11/23-06:15:22.203075 140024953251392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248148.sst): old prealloc_size = 147639500, real_size = 17568176 2024/11/23-06:15:22.203407 140024953251392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248148.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 464631 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0538 GB zip-val = 0.0166 GB avg-val = 115.85 avg-zval = 35.64 2024/11/23-06:15:22.203842 140024953251392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248148.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/11/23-06:15:22.662908 140024953251392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00/248148.sst): second pass time = 0.65's, 82.386'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.46's, 42.802'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2902.842'MB/sec (index lex order gen) rebuild zvType time = 0.39's, 0.295'MB/sec write SST data time = 0.07's, 301.723'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.541'MB, throughput = 18226.817'MB/sec zip my value time = 0.65's, unzip length = 0.054'GB zip my value throughput = 82.386'MB/sec zip pipeline throughput = 82.386'MB/sec entries = 464631 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.85 avg-zval = 35.64 usrkeys = 464631 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.85 avg-zval = 35.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 464631 bytes = 1684304 } seq expand size = 3252417 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0093 GB value = 0.0538 GB dict = 0.54 MB all = 0.0668 GB } __ZipSize{ index = 0.0009 GB value = 0.0166 GB dict = 0.54 MB all = 0.0197 GB } UnZip/Zip{ index = 10.4338 value = 3.2501 dict = 1.00 all = 3.3959 } Zip/UnZip{ index = 0.0958 value = 0.3077 dict = 1.00 all = 0.2945 } ---------------------------- total value len = 1.648868 GB avg = 0.121 KB (by entry num) total key len = 0.381356 GB avg = 0.028 KB total ukey len = 0.272397 GB avg = 0.020 KB total ukey num = 0.013619843 Billion total entry num = 0.013619843 Billion write speed all = 0.026373535 MB/sec (with seq num) write speed all = 0.024958110 MB/sec (without seq num) 2024/11/23-06:15:22.686150 140024953251392 [compaction_job.cc:1948] [default] [JOB 20846] Generated table #248148: 464631 keys, 19681155 bytes, temperature: kWarm 2024/11/23-06:15:22.686221 140024953251392 EVENT_LOG_v1 {"time_micros": 1732313722686189, "cf_name": "default", "job": 20846, "event": "table_file_creation", "file_number": 248148, "file_size": 19681155, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 3829907178, "table_properties": {"data_size": 16561184, "index_size": 890624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13009668, "raw_average_key_size": 28, "raw_value_size": 53825346, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 464631, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732313721, "oldest_key_time": 0, "file_creation_time": 1732313721, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248148, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:15:22.703784 140024953251392 [compaction_job.cc:703] job-20846: subcompact[0], size: 0.540497 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20846/att-00 [248134,248135,248136,248137,248138,248139,248140,248141,248142,248143,248144,248145,248146,248147,248148] 2024/11/23-06:15:22.802504 140024953251392 (Original Log Time 2024/11/23-06:14:33.939217) EVENT_LOG_v1 {"time_micros": 1732313673939178, "job": 20846, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [248095], "files_L3": [248047, 248048, 248049, 248050, 248051, 248052, 248053, 248054, 248055, 248013, 248014, 248015, 248016, 248017, 248127], "score": 0.999869, "input_data_size": 561600007, "oldest_snapshot_seqno": 3830092499} 2024/11/23-06:15:22.802509 140024953251392 (Original Log Time 2024/11/23-06:15:22.801023) [compaction_job.cc:2022] [default] [JOB 20846] Compacted 1@2 + 15@3 files to L3 => 540496592 bytes 2024/11/23-06:15:22.802511 140024953251392 (Original Log Time 2024/11/23-06:15:22.801737) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.5 rd, 11.1 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 508.5 +0.0 blob) out(515.5 +0.0 blob), read-write-amplify(38.8) write-amplify(19.0) OK, records in: 13692118, records dropped: 72275 output_compression: Snappy 2024/11/23-06:15:22.802516 140024953251392 (Original Log Time 2024/11/23-06:15:22.802480) EVENT_LOG_v1 {"time_micros": 1732313722801760, "cf": "default", "job": 20846, "event": "compaction_finished", "compaction_time_micros": 48764574, "compaction_time_cpu_micros": 21333848, "output_level": 3, "num_output_files": 15, "total_output_size": 540496592, "num_input_records": 13692118, "num_output_records": 13619843, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 336348762, "file_range_sync_nanos": 0, "file_fsync_nanos": 13174, "file_prepare_write_nanos": 34016, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-06:15:22.930967 140024953251392 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732313673,"executesMs":48985,"compactionJobId":20846,"attempt":0,"compactionInputRawBytes":1956147213,"compactionInputZipBytes":561600007,"compactionOutputRawBytes":0,"compactionOutputZipBytes":540496592,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b05933fcffb609284b3ab07728db8db0-dd2e434152d9d5b8-00"}}, response = 2024/11/23-06:15:22.931044 140024953251392 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20846/att-00: olev 3, work 48.986 s, result 4.845 ms, install 1.519 ms, input{raw 1.822 GiB zip 535.584 MiB}