2024/11/22-20:12:37.332439 140025045571136 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00: fork to child time = 0.024081 sec 2024/11/22-20:12:37.333537 140025045571136 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00 2024/11/22-20:12:37.341963 140025045571136 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00 2024/11/22-20:12:37.343063 140025045571136 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-20:12:37.343452 140025045571136 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-20:12:37.397932 140025045571136 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09839/att-00/MANIFEST-206347 2024/11/22-20:12:37.430037 140025045571136 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09839/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 225351, last_sequence is 3530673865, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 225167 2024/11/22-20:12:37.430051 140025045571136 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-20:12:37.430218 140025045571136 [version_set.cc:5721] Creating manifest 225351 2024/11/22-20:12:37.471868 140025045571136 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-20:12:37.491112 140025045571136 [compaction_job.cc:2437] [default] [JOB 9839] Compacting 1@2 + 16@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-20:12:37.491143 140025045571136 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 7433 Base level 2, inputs: [225301(27MB)], [225331(34MB) 225332(771KB) 225266(33MB) 225267(33MB) 225268(34MB) 225269(35MB) 225270(35MB) 225271(35MB) 225272(36MB) 225273(36MB) 225274(34MB) 225275(21MB) 225238(35MB) 225239(35MB) 225240(35MB) 225345(37MB)] 2024/11/22-20:12:37.499288 140025045571136 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-20:12:37.499332 140025045571136 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-20:12:37.499360 140025045571136 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-20:12:37.499369 140025045571136 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-20:12:37.499384 140025045571136 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-20:12:38.124529 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225352.sst) /dev/shm/ToplingTemp/Topling-273479-YMT9rp 2024/11/22-20:12:38.698793 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225352.sst): first pass time = 0.57's, 233.741'MB/sec 2024/11/22-20:12:40.629033 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225352.sst): old prealloc_size = 147639500, real_size = 31681264 2024/11/22-20:12:40.629792 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225352.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923928 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 117.27 avg-zval = 32.14 2024/11/22-20:12:40.630636 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225352.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:12:41.559076 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225352.sst): second pass time = 1.93's, 56.178'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 38.462'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2375.210'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.16's, 222.433'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35220.530'MB/sec zip my value time = 1.93's, unzip length = 0.108'GB zip my value throughput = 56.178'MB/sec zip pipeline throughput = 56.178'MB/sec entries = 923928 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.27 avg-zval = 32.14 usrkeys = 923928 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.27 avg-zval = 32.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923928 bytes = 3233760 } seq expand size = 6467496 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5133 value = 3.6490 dict = 1.00 all = 3.7520 } Zip/UnZip{ index = 0.0951 value = 0.2740 dict = 1.00 all = 0.2665 } ---------------------------- total value len = 0.114815 GB avg = 0.124 KB (by entry num) total key len = 0.025870 GB avg = 0.028 KB total ukey len = 0.018479 GB avg = 0.020 KB total ukey num = 0.000923928 Billion total entry num = 0.000923928 Billion write speed all = 0.003446610 MB/sec (with seq num) write speed all = 0.003265529 MB/sec (without seq num) 2024/11/22-20:12:41.594723 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225352: 923928 keys, 35772058 bytes, temperature: kWarm 2024/11/22-20:12:41.594816 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277561594756, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225352, "file_size": 35772058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774766, "largest_seqno": 3530382929, "table_properties": {"data_size": 29692448, "index_size": 1757632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869984, "raw_average_key_size": 28, "raw_value_size": 108347828, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923928, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277558, "oldest_key_time": 0, "file_creation_time": 1732277558, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225352, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:12:41.630847 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225353.sst) /dev/shm/ToplingTemp/Topling-273479-Q26K3m 2024/11/22-20:12:42.459423 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225353.sst): first pass time = 0.83's, 161.993'MB/sec 2024/11/22-20:12:43.908372 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225353.sst): old prealloc_size = 147639500, real_size = 31624880 2024/11/22-20:12:43.908930 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225353.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923898 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0296 GB avg-val = 117.27 avg-zval = 32.08 2024/11/22-20:12:43.909649 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225353.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:12:44.772347 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225353.sst): second pass time = 1.45's, 74.816'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.340'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3277.937'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.314'MB/sec write SST data time = 0.13's, 277.922'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32595.133'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.816'MB/sec zip pipeline throughput = 74.816'MB/sec entries = 923898 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.27 avg-zval = 32.08 usrkeys = 923898 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.27 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923898 bytes = 3233664 } seq expand size = 6467286 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5149 value = 3.6559 dict = 1.00 all = 3.7579 } Zip/UnZip{ index = 0.0951 value = 0.2735 dict = 1.00 all = 0.2661 } ---------------------------- total value len = 0.229631 GB avg = 0.124 KB (by entry num) total key len = 0.051739 GB avg = 0.028 KB total ukey len = 0.036957 GB avg = 0.020 KB total ukey num = 0.001847826 Billion total entry num = 0.001847826 Billion write speed all = 0.006892670 MB/sec (with seq num) write speed all = 0.006530544 MB/sec (without seq num) 2024/11/22-20:12:44.804320 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225353: 923898 keys, 35715778 bytes, temperature: kWarm 2024/11/22-20:12:44.804382 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277564804350, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225353, "file_size": 35715778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774786, "largest_seqno": 3530382944, "table_properties": {"data_size": 29636384, "index_size": 1757312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869144, "raw_average_key_size": 28, "raw_value_size": 108348616, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923898, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277561, "oldest_key_time": 0, "file_creation_time": 1732277561, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225353, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:12:44.839427 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225354.sst) /dev/shm/ToplingTemp/Topling-273479-rfUHmo 2024/11/22-20:12:45.698639 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225354.sst): first pass time = 0.86's, 156.218'MB/sec 2024/11/22-20:12:47.126424 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225354.sst): old prealloc_size = 147639500, real_size = 31729232 2024/11/22-20:12:47.126973 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225354.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923920 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 117.27 avg-zval = 32.16 2024/11/22-20:12:47.127732 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225354.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:12:48.009812 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225354.sst): second pass time = 1.43's, 75.960'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.550'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3376.078'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 282.894'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 22675.061'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.960'MB/sec zip pipeline throughput = 87.453'MB/sec entries = 923920 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.27 avg-zval = 32.16 usrkeys = 923920 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.27 avg-zval = 32.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923920 bytes = 3233728 } seq expand size = 6467440 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.3772 value = 3.6460 dict = 1.00 all = 3.7470 } Zip/UnZip{ index = 0.0964 value = 0.2743 dict = 1.00 all = 0.2669 } ---------------------------- total value len = 0.344447 GB avg = 0.124 KB (by entry num) total key len = 0.077609 GB avg = 0.028 KB total ukey len = 0.055435 GB avg = 0.020 KB total ukey num = 0.002771746 Billion total entry num = 0.002771746 Billion write speed all = 0.010338189 MB/sec (with seq num) write speed all = 0.009795041 MB/sec (without seq num) 2024/11/22-20:12:48.043860 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225354: 923920 keys, 35820066 bytes, temperature: kWarm 2024/11/22-20:12:48.043920 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277568043888, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225354, "file_size": 35820066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774802, "largest_seqno": 3530382957, "table_properties": {"data_size": 29717376, "index_size": 1780672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869760, "raw_average_key_size": 28, "raw_value_size": 108348104, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923920, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277564, "oldest_key_time": 0, "file_creation_time": 1732277564, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225354, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:12:48.082952 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225355.sst) /dev/shm/ToplingTemp/Topling-273479-oXij3p 2024/11/22-20:12:48.908252 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225355.sst): first pass time = 0.83's, 162.636'MB/sec 2024/11/22-20:12:50.517507 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225355.sst): old prealloc_size = 147639500, real_size = 35141808 2024/11/22-20:12:50.518194 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225355.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1082262 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0328 GB avg-val = 96.02 avg-zval = 30.29 2024/11/22-20:12:50.518364 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225355.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:12:51.145006 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225355.sst): second pass time = 1.61's, 64.612'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.666'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3131.974'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.553'MB/sec write SST data time = 0.14's, 263.344'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.032'MB, throughput = 32539.868'MB/sec zip my value time = 1.61's, unzip length = 0.104'GB zip my value throughput = 64.612'MB/sec zip pipeline throughput = 102.631'MB/sec entries = 1082262 avg-key = 28.00 avg-zkey = 1.93 avg-val = 96.02 avg-zval = 30.29 usrkeys = 1082262 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.24 avg-zval = 30.29 TagRS{ kinds = RS_Key0_TagN bytes = 169200 } TagArray{ size = 35029 bytes = 100720 } seq expand size = 245203 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0328 GB dict = 1.03 MB all = 0.0362 GB } UnZip/Zip{ index = 10.3799 value = 3.1695 dict = 1.00 all = 3.7092 } Zip/UnZip{ index = 0.0963 value = 0.3155 dict = 1.00 all = 0.2696 } ---------------------------- total value len = 0.448606 GB avg = 0.116 KB (by entry num) total key len = 0.107912 GB avg = 0.028 KB total ukey len = 0.077080 GB avg = 0.020 KB total ukey num = 0.003854008 Billion total entry num = 0.003854008 Billion write speed all = 0.013630792 MB/sec (with seq num) write speed all = 0.012875623 MB/sec (without seq num) 2024/11/22-20:12:51.178053 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225355: 1082262 keys, 36184697 bytes, temperature: kWarm 2024/11/22-20:12:51.178117 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277571178082, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225355, "file_size": 36184697, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3530382984, "table_properties": {"data_size": 32785760, "index_size": 2085312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30303336, "raw_average_key_size": 28, "raw_value_size": 103914441, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1082262, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277568, "oldest_key_time": 0, "file_creation_time": 1732277568, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225355, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:12:51.219694 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225356.sst) /dev/shm/ToplingTemp/Topling-273479-nYt8go 2024/11/22-20:12:52.045656 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225356.sst): first pass time = 0.83's, 162.506'MB/sec 2024/11/22-20:12:53.734807 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225356.sst): old prealloc_size = 147639500, real_size = 34791440 2024/11/22-20:12:53.735328 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225356.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922119 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0328 GB avg-val = 117.55 avg-zval = 35.58 2024/11/22-20:12:53.735882 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225356.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:12:54.578640 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225356.sst): second pass time = 1.69's, 64.198'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.833'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3528.954'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 295.105'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34219.387'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.198'MB/sec zip pipeline throughput = 119.348'MB/sec entries = 922119 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.55 avg-zval = 35.58 usrkeys = 922119 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.96 avg-zval = 35.58 TagRS{ kinds = RS_Key0_TagN bytes = 144200 } TagArray{ size = 580949 bytes = 2033344 } seq expand size = 4066643 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5134 value = 3.3042 dict = 1.00 all = 3.5478 } Zip/UnZip{ index = 0.0951 value = 0.3026 dict = 1.00 all = 0.2819 } ---------------------------- total value len = 0.561072 GB avg = 0.117 KB (by entry num) total key len = 0.133732 GB avg = 0.028 KB total ukey len = 0.095523 GB avg = 0.020 KB total ukey num = 0.004776127 Billion total entry num = 0.004776127 Billion write speed all = 0.017016358 MB/sec (with seq num) write speed all = 0.016080585 MB/sec (without seq num) 2024/11/22-20:12:54.612753 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225356: 922119 keys, 37831340 bytes, temperature: kWarm 2024/11/22-20:12:54.612822 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277574612788, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225356, "file_size": 37831340, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3493136006, "table_properties": {"data_size": 32806528, "index_size": 1754176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25819332, "raw_average_key_size": 28, "raw_value_size": 108398518, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922119, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277571, "oldest_key_time": 0, "file_creation_time": 1732277571, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225356, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:12:54.651705 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225357.sst) /dev/shm/ToplingTemp/Topling-273479-r0cf6p 2024/11/22-20:12:55.527166 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225357.sst): first pass time = 0.88's, 153.320'MB/sec 2024/11/22-20:12:56.990938 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225357.sst): old prealloc_size = 147639500, real_size = 33531904 2024/11/22-20:12:56.991459 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225357.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923588 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0315 GB avg-val = 117.32 avg-zval = 34.15 2024/11/22-20:12:56.992242 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225357.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:12:57.925498 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225357.sst): second pass time = 1.46's, 74.059'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.257'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3523.498'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 295.264'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34059.577'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.059'MB/sec zip pipeline throughput = 93.510'MB/sec entries = 923588 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.32 avg-zval = 34.15 usrkeys = 923588 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.32 avg-zval = 34.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923588 bytes = 3232576 } seq expand size = 6465116 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5106 value = 3.4352 dict = 1.00 all = 3.5675 } Zip/UnZip{ index = 0.0951 value = 0.2911 dict = 1.00 all = 0.2803 } ---------------------------- total value len = 0.675894 GB avg = 0.119 KB (by entry num) total key len = 0.159592 GB avg = 0.028 KB total ukey len = 0.113994 GB avg = 0.020 KB total ukey num = 0.005699715 Billion total entry num = 0.005699715 Billion write speed all = 0.020460132 MB/sec (with seq num) write speed all = 0.019343494 MB/sec (without seq num) 2024/11/22-20:12:57.957666 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225357: 923588 keys, 37622260 bytes, temperature: kWarm 2024/11/22-20:12:57.957742 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277577957707, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225357, "file_size": 37622260, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3394785136, "largest_seqno": 3530382616, "table_properties": {"data_size": 31543408, "index_size": 1757440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25860464, "raw_average_key_size": 28, "raw_value_size": 108357276, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923588, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277574, "oldest_key_time": 0, "file_creation_time": 1732277574, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225357, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:12:57.999303 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225358.sst) /dev/shm/ToplingTemp/Topling-273479-cdxxUp 2024/11/22-20:12:58.821296 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225358.sst): first pass time = 0.82's, 163.292'MB/sec 2024/11/22-20:13:00.305377 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225358.sst): old prealloc_size = 147639500, real_size = 33341792 2024/11/22-20:13:00.305922 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225358.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923716 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0314 GB avg-val = 117.30 avg-zval = 33.94 2024/11/22-20:13:00.307614 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225358.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:13:01.240563 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225358.sst): second pass time = 1.48's, 73.044'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.027'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3333.733'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 291.377'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32144.715'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.044'MB/sec zip pipeline throughput = 93.400'MB/sec entries = 923716 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.30 avg-zval = 33.94 usrkeys = 923716 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.30 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923716 bytes = 3233024 } seq expand size = 6466012 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5136 value = 3.4559 dict = 1.00 all = 3.5856 } Zip/UnZip{ index = 0.0951 value = 0.2894 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 0.790714 GB avg = 0.119 KB (by entry num) total key len = 0.185456 GB avg = 0.028 KB total ukey len = 0.132469 GB avg = 0.020 KB total ukey num = 0.006623431 Billion total entry num = 0.006623431 Billion write speed all = 0.023903384 MB/sec (with seq num) write speed all = 0.022605885 MB/sec (without seq num) 2024/11/22-20:13:01.274822 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225358: 923716 keys, 37432346 bytes, temperature: kWarm 2024/11/22-20:13:01.274885 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277581274853, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225358, "file_size": 37432346, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3530382781, "table_properties": {"data_size": 31353488, "index_size": 1757184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25864048, "raw_average_key_size": 28, "raw_value_size": 108353796, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923716, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277577, "oldest_key_time": 0, "file_creation_time": 1732277577, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225358, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:13:01.319734 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225359.sst) /dev/shm/ToplingTemp/Topling-273479-oNzbZo 2024/11/22-20:13:02.144819 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225359.sst): first pass time = 0.83's, 162.680'MB/sec 2024/11/22-20:13:03.619097 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225359.sst): old prealloc_size = 147639500, real_size = 33987584 2024/11/22-20:13:03.619640 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225359.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925669 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0320 GB avg-val = 117.00 avg-zval = 34.56 2024/11/22-20:13:03.621310 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225359.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:13:04.546953 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225359.sst): second pass time = 1.47's, 73.514'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 41.044'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3379.240'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 296.114'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30118.424'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.514'MB/sec zip pipeline throughput = 97.668'MB/sec entries = 925669 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.00 avg-zval = 34.56 usrkeys = 925669 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.00 avg-zval = 34.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925669 bytes = 3239856 } seq expand size = 6479683 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5121 value = 3.3849 dict = 1.00 all = 3.5244 } Zip/UnZip{ index = 0.0951 value = 0.2954 dict = 1.00 all = 0.2837 } ---------------------------- total value len = 0.905493 GB avg = 0.120 KB (by entry num) total key len = 0.211375 GB avg = 0.028 KB total ukey len = 0.150982 GB avg = 0.020 KB total ukey num = 0.007549100 Billion total entry num = 0.007549100 Billion write speed all = 0.027346416 MB/sec (with seq num) write speed all = 0.025867703 MB/sec (without seq num) 2024/11/22-20:13:04.581482 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225359: 925669 keys, 38082186 bytes, temperature: kWarm 2024/11/22-20:13:04.581546 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277584581514, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225359, "file_size": 38082186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3530382813, "table_properties": {"data_size": 31994800, "index_size": 1761152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25918732, "raw_average_key_size": 28, "raw_value_size": 108299054, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925669, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277581, "oldest_key_time": 0, "file_creation_time": 1732277581, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225359, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:13:04.632351 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225360.sst) /dev/shm/ToplingTemp/Topling-273479-XtnsQm 2024/11/22-20:13:05.498530 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225360.sst): first pass time = 0.87's, 154.962'MB/sec 2024/11/22-20:13:06.951724 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225360.sst): old prealloc_size = 147639500, real_size = 34205472 2024/11/22-20:13:06.952257 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225360.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927431 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0322 GB avg-val = 116.72 avg-zval = 34.72 2024/11/22-20:13:06.954265 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225360.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:13:07.865658 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225360.sst): second pass time = 1.45's, 74.529'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.913'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3498.313'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 296.333'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33543.599'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.529'MB/sec zip pipeline throughput = 75.713'MB/sec entries = 927431 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.72 avg-zval = 34.72 usrkeys = 927431 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.72 avg-zval = 34.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927431 bytes = 3246016 } seq expand size = 6492017 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4444 value = 3.3621 dict = 1.00 all = 3.5039 } Zip/UnZip{ index = 0.0957 value = 0.2974 dict = 1.00 all = 0.2854 } ---------------------------- total value len = 1.020234 GB avg = 0.120 KB (by entry num) total key len = 0.237343 GB avg = 0.028 KB total ukey len = 0.169531 GB avg = 0.020 KB total ukey num = 0.008476531 Billion total entry num = 0.008476531 Billion write speed all = 0.030789185 MB/sec (with seq num) write speed all = 0.029128942 MB/sec (without seq num) 2024/11/22-20:13:07.900133 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225360: 927431 keys, 38305354 bytes, temperature: kWarm 2024/11/22-20:13:07.900195 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277587900162, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225360, "file_size": 38305354, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3530383246, "table_properties": {"data_size": 32197520, "index_size": 1775936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25968068, "raw_average_key_size": 28, "raw_value_size": 108249770, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277584, "oldest_key_time": 0, "file_creation_time": 1732277584, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225360, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:13:07.943822 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225361.sst) /dev/shm/ToplingTemp/Topling-273479-RJ9kYm 2024/11/22-20:13:08.742045 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225361.sst): first pass time = 0.80's, 168.154'MB/sec 2024/11/22-20:13:10.176325 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225361.sst): old prealloc_size = 147639500, real_size = 33003168 2024/11/22-20:13:10.176874 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225361.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923742 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0310 GB avg-val = 117.30 avg-zval = 33.58 2024/11/22-20:13:10.177638 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225361.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:13:11.093290 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225361.sst): second pass time = 1.43's, 75.578'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.453'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3324.911'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 288.621'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34476.934'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.578'MB/sec zip pipeline throughput = 87.428'MB/sec entries = 923742 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.30 avg-zval = 33.58 usrkeys = 923742 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.30 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923742 bytes = 3233104 } seq expand size = 6466194 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5147 value = 3.4936 dict = 1.00 all = 3.6184 } Zip/UnZip{ index = 0.0951 value = 0.2862 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 1.135054 GB avg = 0.121 KB (by entry num) total key len = 0.263208 GB avg = 0.028 KB total ukey len = 0.188005 GB avg = 0.020 KB total ukey num = 0.009400273 Billion total entry num = 0.009400273 Billion write speed all = 0.034230839 MB/sec (with seq num) write speed all = 0.032389814 MB/sec (without seq num) 2024/11/22-20:13:11.126472 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225361: 923742 keys, 37093314 bytes, temperature: kWarm 2024/11/22-20:13:11.126536 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277591126506, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225361, "file_size": 37093314, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701165, "largest_seqno": 3530381643, "table_properties": {"data_size": 31014992, "index_size": 1757056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25864776, "raw_average_key_size": 28, "raw_value_size": 108353060, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923742, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277587, "oldest_key_time": 0, "file_creation_time": 1732277587, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225361, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:13:11.164179 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225362.sst) /dev/shm/ToplingTemp/Topling-273479-SBvPkn 2024/11/22-20:13:12.271087 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225362.sst): first pass time = 1.11's, 121.259'MB/sec 2024/11/22-20:13:13.606775 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225362.sst): old prealloc_size = 147639500, real_size = 32754928 2024/11/22-20:13:13.607318 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225362.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924067 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.25 avg-zval = 33.28 2024/11/22-20:13:13.608071 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225362.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:13:14.522892 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225362.sst): second pass time = 1.33's, 81.175'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.220'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3476.173'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 289.008'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34023.652'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.175'MB/sec zip pipeline throughput = 81.175'MB/sec entries = 924067 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.25 avg-zval = 33.28 usrkeys = 924067 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.25 avg-zval = 33.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924067 bytes = 3234256 } seq expand size = 6468469 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4434 value = 3.5229 dict = 1.00 all = 3.6427 } Zip/UnZip{ index = 0.0958 value = 0.2839 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 1.249866 GB avg = 0.121 KB (by entry num) total key len = 0.289082 GB avg = 0.028 KB total ukey len = 0.206487 GB avg = 0.020 KB total ukey num = 0.010324340 Billion total entry num = 0.010324340 Billion write speed all = 0.037671816 MB/sec (with seq num) write speed all = 0.035649984 MB/sec (without seq num) 2024/11/22-20:13:14.558811 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225362: 924067 keys, 36845830 bytes, temperature: kWarm 2024/11/22-20:13:14.558875 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277594558842, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225362, "file_size": 36845830, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3388810268, "largest_seqno": 3530382164, "table_properties": {"data_size": 30754080, "index_size": 1769664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25873876, "raw_average_key_size": 28, "raw_value_size": 108343874, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924067, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277591, "oldest_key_time": 0, "file_creation_time": 1732277591, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225362, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:13:14.596044 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225363.sst) /dev/shm/ToplingTemp/Topling-273479-9Mx6fq 2024/11/22-20:13:15.450197 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225363.sst): first pass time = 0.85's, 157.143'MB/sec 2024/11/22-20:13:16.786982 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225363.sst): old prealloc_size = 147639500, real_size = 32664912 2024/11/22-20:13:16.787506 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225363.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923439 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0307 GB avg-val = 117.35 avg-zval = 33.22 2024/11/22-20:13:16.789210 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225363.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:13:17.696189 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225363.sst): second pass time = 1.34's, 81.098'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.425'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3496.223'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 286.942'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33402.765'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.098'MB/sec zip pipeline throughput = 81.098'MB/sec entries = 923439 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.35 avg-zval = 33.22 usrkeys = 923439 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.35 avg-zval = 33.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923439 bytes = 3232048 } seq expand size = 6464073 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5116 value = 3.5324 dict = 1.00 all = 3.6518 } Zip/UnZip{ index = 0.0951 value = 0.2831 dict = 1.00 all = 0.2738 } ---------------------------- total value len = 1.364691 GB avg = 0.121 KB (by entry num) total key len = 0.314938 GB avg = 0.028 KB total ukey len = 0.224956 GB avg = 0.020 KB total ukey num = 0.011247779 Billion total entry num = 0.011247779 Billion write speed all = 0.041112367 MB/sec (with seq num) write speed all = 0.038909867 MB/sec (without seq num) 2024/11/22-20:13:17.731262 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225363: 923439 keys, 36754338 bytes, temperature: kWarm 2024/11/22-20:13:17.731326 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277597731293, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225363, "file_size": 36754338, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3530382746, "table_properties": {"data_size": 30676864, "index_size": 1756992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25856292, "raw_average_key_size": 28, "raw_value_size": 108361522, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923439, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277594, "oldest_key_time": 0, "file_creation_time": 1732277594, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225363, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:13:17.767421 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225364.sst) /dev/shm/ToplingTemp/Topling-273479-M9RRHq 2024/11/22-20:13:18.639147 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225364.sst): first pass time = 0.87's, 153.976'MB/sec 2024/11/22-20:13:20.138262 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225364.sst): old prealloc_size = 147639500, real_size = 36117584 2024/11/22-20:13:20.138851 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225364.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929792 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0341 GB avg-val = 116.35 avg-zval = 36.68 2024/11/22-20:13:20.139145 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225364.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:13:20.855023 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225364.sst): second pass time = 1.50's, 72.194'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 52.820'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3117.456'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.395'MB/sec write SST data time = 0.13's, 297.397'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27288.464'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.194'MB/sec zip pipeline throughput = 72.194'MB/sec entries = 929792 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.35 avg-zval = 36.68 usrkeys = 929792 avg-key = 20.00 avg-zkey = 1.92 avg-val = 118.13 avg-zval = 36.68 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 236500 bytes = 739072 } seq expand size = 1655500 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4432 value = 3.1721 dict = 1.00 all = 3.5453 } Zip/UnZip{ index = 0.0958 value = 0.3152 dict = 1.00 all = 0.2821 } ---------------------------- total value len = 1.474531 GB avg = 0.121 KB (by entry num) total key len = 0.340972 GB avg = 0.028 KB total ukey len = 0.243551 GB avg = 0.020 KB total ukey num = 0.012177571 Billion total entry num = 0.012177571 Billion write speed all = 0.044434709 MB/sec (with seq num) write speed all = 0.042050325 MB/sec (without seq num) 2024/11/22-20:13:20.888484 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225364: 929792 keys, 37857991 bytes, temperature: kWarm 2024/11/22-20:13:20.888549 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277600888513, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225364, "file_size": 37857991, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3530383040, "table_properties": {"data_size": 34104256, "index_size": 1780672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26034176, "raw_average_key_size": 28, "raw_value_size": 108183652, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929792, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277597, "oldest_key_time": 0, "file_creation_time": 1732277597, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225364, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:13:20.925710 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225365.sst) /dev/shm/ToplingTemp/Topling-273479-PSiMVo 2024/11/22-20:13:21.787899 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225365.sst): first pass time = 0.86's, 155.678'MB/sec 2024/11/22-20:13:23.105948 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225365.sst): old prealloc_size = 147639500, real_size = 31828128 2024/11/22-20:13:23.106530 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225365.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923732 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0298 GB avg-val = 117.30 avg-zval = 32.30 2024/11/22-20:13:23.108198 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225365.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:13:23.976957 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225365.sst): second pass time = 1.32's, 82.274'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.405'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3145.477'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 267.432'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31163.443'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.274'MB/sec zip pipeline throughput = 82.274'MB/sec entries = 923732 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.30 avg-zval = 32.30 usrkeys = 923732 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.29 avg-zval = 32.30 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 922725 bytes = 3229552 } seq expand size = 6459075 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5130 value = 3.6312 dict = 1.00 all = 3.7217 } Zip/UnZip{ index = 0.0951 value = 0.2754 dict = 1.00 all = 0.2687 } ---------------------------- total value len = 1.589343 GB avg = 0.121 KB (by entry num) total key len = 0.366836 GB avg = 0.028 KB total ukey len = 0.262026 GB avg = 0.020 KB total ukey num = 0.013101303 Billion total entry num = 0.013101303 Billion write speed all = 0.047874138 MB/sec (with seq num) write speed all = 0.045309083 MB/sec (without seq num) 2024/11/22-20:13:24.010354 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225365: 923732 keys, 36063306 bytes, temperature: kWarm 2024/11/22-20:13:24.010415 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277604010384, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225365, "file_size": 36063306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3530383050, "table_properties": {"data_size": 29839696, "index_size": 1757312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25864496, "raw_average_key_size": 28, "raw_value_size": 108353248, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923732, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277600, "oldest_key_time": 0, "file_creation_time": 1732277600, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225365, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:13:24.048186 140025045571136 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225366.sst) /dev/shm/ToplingTemp/Topling-273479-FpnFOo 2024/11/22-20:13:24.476854 140025045571136 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225366.sst): first pass time = 0.43's, 240.384'MB/sec 2024/11/22-20:13:25.537586 140025045571136 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225366.sst): old prealloc_size = 147639500, real_size = 27587744 2024/11/22-20:13:25.538044 140025045571136 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225366.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 712078 prefix = 4 raw-key = 0.0114 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0831 GB zip-val = 0.0261 GB avg-val = 116.70 avg-zval = 36.59 2024/11/22-20:13:25.538651 140025045571136 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225366.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0018 GB, waited 0.000 sec, Key+Value = 0.103 GB 2024/11/22-20:13:26.212054 140025045571136 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00/225366.sst): second pass time = 1.06's, 78.431'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.67's, 45.579'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3120.614'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.310'MB/sec write SST data time = 0.10's, 310.055'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.834'MB, throughput = 26075.343'MB/sec zip my value time = 1.06's, unzip length = 0.083'GB zip my value throughput = 78.431'MB/sec zip pipeline throughput = 78.431'MB/sec entries = 712078 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.70 avg-zval = 36.59 usrkeys = 712078 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.70 avg-zval = 36.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 712078 bytes = 2492288 } seq expand size = 4984546 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0142 GB value = 0.0831 GB dict = 0.83 MB all = 0.1030 GB } __ZipSize{ index = 0.0014 GB value = 0.0261 GB dict = 0.83 MB all = 0.0307 GB } UnZip/Zip{ index = 10.5103 value = 3.1894 dict = 1.00 all = 3.3517 } Zip/UnZip{ index = 0.0951 value = 0.3135 dict = 1.00 all = 0.2984 } ---------------------------- total value len = 1.677425 GB avg = 0.121 KB (by entry num) total key len = 0.386775 GB avg = 0.028 KB total ukey len = 0.276268 GB avg = 0.020 KB total ukey num = 0.013813381 Billion total entry num = 0.013813381 Billion write speed all = 0.050514980 MB/sec (with seq num) write speed all = 0.047810657 MB/sec (without seq num) 2024/11/22-20:13:26.240857 140025045571136 [compaction_job.cc:1948] [default] [JOB 9839] Generated table #225366: 712078 keys, 30740840 bytes, temperature: kWarm 2024/11/22-20:13:26.240923 140025045571136 EVENT_LOG_v1 {"time_micros": 1732277606240891, "cf_name": "default", "job": 9839, "event": "table_file_creation", "file_number": 225366, "file_size": 30740840, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701550, "largest_seqno": 3530383087, "table_properties": {"data_size": 26054544, "index_size": 1355008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19938184, "raw_average_key_size": 28, "raw_value_size": 83097300, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 712078, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277604, "oldest_key_time": 0, "file_creation_time": 1732277604, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225366, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:13:26.264421 140025045571136 [compaction_job.cc:703] job-09839: subcompact[0], size: 0.548122 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09839/att-00 [225352,225353,225354,225355,225356,225357,225358,225359,225360,225361,225362,225363,225364,225365,225366] 2024/11/22-20:13:26.367871 140025045571136 (Original Log Time 2024/11/22-20:12:37.491228) EVENT_LOG_v1 {"time_micros": 1732277557491164, "job": 9839, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [225301], "files_L3": [225331, 225332, 225266, 225267, 225268, 225269, 225270, 225271, 225272, 225273, 225274, 225275, 225238, 225239, 225240, 225345], "score": 0.999088, "input_data_size": 571117364, "oldest_snapshot_seqno": 3530673826} 2024/11/22-20:13:26.367875 140025045571136 (Original Log Time 2024/11/22-20:13:26.363730) [compaction_job.cc:2022] [default] [JOB 9839] Compacted 1@2 + 16@3 files to L3 => 548121704 bytes 2024/11/22-20:13:26.367877 140025045571136 (Original Log Time 2024/11/22-20:13:26.367343) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.7 rd, 11.2 wr, level 3, files in(1, 16) out(15 +0 blob) MB in(27.1, 517.6 +0.0 blob) out(522.7 +0.0 blob), read-write-amplify(39.4) write-amplify(19.3) OK, records in: 13904084, records dropped: 90703 output_compression: Snappy 2024/11/22-20:13:26.367883 140025045571136 (Original Log Time 2024/11/22-20:13:26.367846) EVENT_LOG_v1 {"time_micros": 1732277606367367, "cf": "default", "job": 9839, "event": "compaction_finished", "compaction_time_micros": 48773202, "compaction_time_cpu_micros": 21197384, "output_level": 3, "num_output_files": 15, "total_output_size": 548121704, "num_input_records": 13904084, "num_output_records": 13813381, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 347951808, "file_range_sync_nanos": 0, "file_fsync_nanos": 14293, "file_prepare_write_nanos": 32497, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/22-20:13:26.500559 140025045571136 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732277557,"executesMs":49026,"compactionJobId":9839,"attempt":0,"compactionInputRawBytes":1994964553,"compactionInputZipBytes":571117364,"compactionOutputRawBytes":0,"compactionOutputZipBytes":548121704,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-2a445287efec5243aeb59a8228cfee8e-36d973b2da6c6a00-00"}}, response = 2024/11/22-20:13:26.500635 140025045571136 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09839/att-00: olev 3, work 49.026 s, result 4.912 ms, install 4.175 ms, input{raw 1.858 GiB zip 544.660 MiB}