2024/11/22-16:25:26.967751 140025205032512 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00: fork to child time = 0.009031 sec 2024/11/22-16:25:26.968523 140025205032512 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00 2024/11/22-16:25:26.975863 140025205032512 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00 2024/11/22-16:25:26.976974 140025205032512 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-16:25:26.977246 140025205032512 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-16:25:27.023797 140025205032512 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04473/att-00/MANIFEST-206347 2024/11/22-16:25:27.055729 140025205032512 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04473/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 214400, last_sequence is 3377457678, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 214205 2024/11/22-16:25:27.055735 140025205032512 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-16:25:27.055967 140025205032512 [version_set.cc:5721] Creating manifest 214400 2024/11/22-16:25:27.087966 140025205032512 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-16:25:27.107937 140025205032512 [compaction_job.cc:2437] [default] [JOB 4473] Compacting 1@2 + 12@3 files to L3, score 0.95, subcompactions 1 : 1 2024/11/22-16:25:27.107954 140025205032512 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 3486 Base level 2, inputs: [214357(27MB)], [214397(36MB) 214398(5999KB) 214308(36MB) 214309(11MB) 214279(36MB) 214280(35MB) 214281(34MB) 214282(36MB) 214285(36MB) 214286(37MB) 214287(36MB) 214376(36MB)] 2024/11/22-16:25:27.113836 140025205032512 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:25:27.113873 140025205032512 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:25:27.113891 140025205032512 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:25:27.113896 140025205032512 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:25:27.113905 140025205032512 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:25:27.597392 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214401.sst) /dev/shm/ToplingTemp/Topling-197490-94Ttcv 2024/11/22-16:25:28.193162 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214401.sst): first pass time = 0.60's, 225.299'MB/sec 2024/11/22-16:25:29.806860 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214401.sst): old prealloc_size = 147639500, real_size = 34147824 2024/11/22-16:25:29.807458 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214401.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922929 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0321 GB avg-val = 117.43 avg-zval = 34.83 2024/11/22-16:25:29.808299 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214401.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:25:30.920008 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214401.sst): second pass time = 1.61's, 67.196'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 34.246'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3066.704'MB/sec (index lex order gen) rebuild zvType time = 0.96's, 0.240'MB/sec write SST data time = 0.15's, 250.647'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34332.657'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.196'MB/sec zip pipeline throughput = 67.196'MB/sec entries = 922929 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.43 avg-zval = 34.83 usrkeys = 922929 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.43 avg-zval = 34.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922929 bytes = 3114896 } seq expand size = 6460503 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4442 value = 3.3710 dict = 1.00 all = 3.5209 } Zip/UnZip{ index = 0.0957 value = 0.2966 dict = 1.00 all = 0.2840 } ---------------------------- total value len = 0.114836 GB avg = 0.124 KB (by entry num) total key len = 0.025842 GB avg = 0.028 KB total ukey len = 0.018459 GB avg = 0.020 KB total ukey num = 0.000922929 Billion total entry num = 0.000922929 Billion write speed all = 0.005174311 MB/sec (with seq num) write speed all = 0.004902740 MB/sec (without seq num) 2024/11/22-16:25:30.954142 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214401: 922929 keys, 38120564 bytes, temperature: kWarm 2024/11/22-16:25:30.954224 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263930954174, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214401, "file_size": 38120564, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701396, "largest_seqno": 3376772517, "table_properties": {"data_size": 32149536, "index_size": 1767360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842012, "raw_average_key_size": 28, "raw_value_size": 108375750, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922929, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263927, "oldest_key_time": 0, "file_creation_time": 1732263927, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214401, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:25:30.991157 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214402.sst) /dev/shm/ToplingTemp/Topling-197490-PuSfTr 2024/11/22-16:25:32.058272 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214402.sst): first pass time = 1.07's, 125.780'MB/sec 2024/11/22-16:25:33.392201 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214402.sst): old prealloc_size = 147639500, real_size = 33834512 2024/11/22-16:25:33.392757 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214402.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921166 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.70 avg-zval = 34.58 2024/11/22-16:25:33.393515 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214402.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:25:34.319914 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214402.sst): second pass time = 1.33's, 81.367'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.749'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3271.697'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.287'MB/sec write SST data time = 0.12's, 306.319'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31653.698'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.367'MB/sec zip pipeline throughput = 81.367'MB/sec entries = 921166 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.70 avg-zval = 34.58 usrkeys = 921166 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.70 avg-zval = 34.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921166 bytes = 3108944 } seq expand size = 6448162 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5121 value = 3.4041 dict = 1.00 all = 3.5505 } Zip/UnZip{ index = 0.0951 value = 0.2938 dict = 1.00 all = 0.2817 } ---------------------------- total value len = 0.229710 GB avg = 0.125 KB (by entry num) total key len = 0.051635 GB avg = 0.028 KB total ukey len = 0.036882 GB avg = 0.020 KB total ukey num = 0.001844095 Billion total entry num = 0.001844095 Billion write speed all = 0.010346878 MB/sec (with seq num) write speed all = 0.009804322 MB/sec (without seq num) 2024/11/22-16:25:34.353099 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214402: 921166 keys, 37802688 bytes, temperature: kWarm 2024/11/22-16:25:34.353153 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263934353123, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214402, "file_size": 37802688, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701431, "largest_seqno": 3376772537, "table_properties": {"data_size": 31851456, "index_size": 1752576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25792648, "raw_average_key_size": 28, "raw_value_size": 108425188, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921166, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263930, "oldest_key_time": 0, "file_creation_time": 1732263930, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214402, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:25:34.389555 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214403.sst) /dev/shm/ToplingTemp/Topling-197490-BRTKLu 2024/11/22-16:25:35.295383 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214403.sst): first pass time = 0.91's, 148.177'MB/sec 2024/11/22-16:25:36.611972 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214403.sst): old prealloc_size = 147639500, real_size = 34582816 2024/11/22-16:25:36.612485 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214403.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923071 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0326 GB avg-val = 117.40 avg-zval = 35.31 2024/11/22-16:25:36.613225 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214403.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:25:37.509053 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214403.sst): second pass time = 1.32's, 82.360'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.980'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3574.427'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.13's, 304.800'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34410.208'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.360'MB/sec zip pipeline throughput = 82.360'MB/sec entries = 923071 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.40 avg-zval = 35.31 usrkeys = 923071 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.40 avg-zval = 35.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923071 bytes = 3115376 } seq expand size = 6461497 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5151 value = 3.3247 dict = 1.00 all = 3.4811 } Zip/UnZip{ index = 0.0951 value = 0.3008 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 0.344543 GB avg = 0.125 KB (by entry num) total key len = 0.077481 GB avg = 0.028 KB total ukey len = 0.055343 GB avg = 0.020 KB total ukey num = 0.002767166 Billion total entry num = 0.002767166 Billion write speed all = 0.015518761 MB/sec (with seq num) write speed all = 0.014704721 MB/sec (without seq num) 2024/11/22-16:25:37.546806 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214403: 923071 keys, 38556018 bytes, temperature: kWarm 2024/11/22-16:25:37.546865 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263937546834, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214403, "file_size": 38556018, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701445, "largest_seqno": 3376772566, "table_properties": {"data_size": 32596176, "index_size": 1755712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25845988, "raw_average_key_size": 28, "raw_value_size": 108371842, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923071, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263934, "oldest_key_time": 0, "file_creation_time": 1732263934, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214403, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:25:37.583942 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214404.sst) /dev/shm/ToplingTemp/Topling-197490-sVoKGu 2024/11/22-16:25:38.475345 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214404.sst): first pass time = 0.89's, 150.575'MB/sec 2024/11/22-16:25:39.789655 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214404.sst): old prealloc_size = 147639500, real_size = 33924448 2024/11/22-16:25:39.790201 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214404.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921339 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.68 avg-zval = 34.67 2024/11/22-16:25:39.790933 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214404.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:25:40.712831 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214404.sst): second pass time = 1.31's, 82.538'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.047'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3333.095'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.13's, 298.461'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 29519.599'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.538'MB/sec zip pipeline throughput = 82.538'MB/sec entries = 921339 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.68 avg-zval = 34.67 usrkeys = 921339 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.68 avg-zval = 34.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921339 bytes = 3109536 } seq expand size = 6449373 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5130 value = 3.3944 dict = 1.00 all = 3.5420 } Zip/UnZip{ index = 0.0951 value = 0.2946 dict = 1.00 all = 0.2823 } ---------------------------- total value len = 0.459413 GB avg = 0.125 KB (by entry num) total key len = 0.103278 GB avg = 0.028 KB total ukey len = 0.073770 GB avg = 0.020 KB total ukey num = 0.003688505 Billion total entry num = 0.003688505 Billion write speed all = 0.020688974 MB/sec (with seq num) write speed all = 0.019604025 MB/sec (without seq num) 2024/11/22-16:25:40.747599 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214404: 921339 keys, 37893118 bytes, temperature: kWarm 2024/11/22-16:25:40.747660 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263940747626, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214404, "file_size": 37893118, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701473, "largest_seqno": 3376772597, "table_properties": {"data_size": 31941136, "index_size": 1752768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25797492, "raw_average_key_size": 28, "raw_value_size": 108420334, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921339, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263937, "oldest_key_time": 0, "file_creation_time": 1732263937, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214404, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:25:40.780855 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214405.sst) /dev/shm/ToplingTemp/Topling-197490-Tx3Dst 2024/11/22-16:25:41.644057 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214405.sst): first pass time = 0.86's, 155.494'MB/sec 2024/11/22-16:25:43.037056 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214405.sst): old prealloc_size = 147639500, real_size = 35159200 2024/11/22-16:25:43.037662 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214405.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1082903 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0328 GB avg-val = 95.94 avg-zval = 30.30 2024/11/22-16:25:43.037898 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214405.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:25:43.669136 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214405.sst): second pass time = 1.39's, 74.619'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.437'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3557.044'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.550'MB/sec write SST data time = 0.14's, 260.813'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.033'MB, throughput = 33442.157'MB/sec zip my value time = 1.39's, unzip length = 0.104'GB zip my value throughput = 74.619'MB/sec zip pipeline throughput = 74.619'MB/sec entries = 1082903 avg-key = 28.00 avg-zkey = 1.91 avg-val = 95.94 avg-zval = 30.30 usrkeys = 1082903 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.38 avg-zval = 30.30 TagRS{ kinds = RS_Key0_TagN bytes = 169320 } TagArray{ size = 67343 bytes = 202048 } seq expand size = 471401 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0217 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0328 GB dict = 1.03 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4472 value = 3.1661 dict = 1.00 all = 3.6971 } Zip/UnZip{ index = 0.0957 value = 0.3158 dict = 1.00 all = 0.2705 } ---------------------------- total value len = 0.563781 GB avg = 0.118 KB (by entry num) total key len = 0.133599 GB avg = 0.028 KB total ukey len = 0.095428 GB avg = 0.020 KB total ukey num = 0.004771408 Billion total entry num = 0.004771408 Billion write speed all = 0.025638433 MB/sec (with seq num) write speed all = 0.024235107 MB/sec (without seq num) 2024/11/22-16:25:43.700682 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214405: 1082903 keys, 36303961 bytes, temperature: kWarm 2024/11/22-16:25:43.700744 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263943700713, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214405, "file_size": 36303961, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3376769819, "table_properties": {"data_size": 32815184, "index_size": 2073088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30321284, "raw_average_key_size": 28, "raw_value_size": 103896587, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1082903, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263940, "oldest_key_time": 0, "file_creation_time": 1732263940, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214405, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:25:43.735632 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214406.sst) /dev/shm/ToplingTemp/Topling-197490-ospm0u 2024/11/22-16:25:44.509685 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214406.sst): first pass time = 0.77's, 173.405'MB/sec 2024/11/22-16:25:45.850684 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214406.sst): old prealloc_size = 147639500, real_size = 35011136 2024/11/22-16:25:45.851275 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214406.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922592 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0330 GB avg-val = 117.48 avg-zval = 35.77 2024/11/22-16:25:45.851790 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214406.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:25:46.682094 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214406.sst): second pass time = 1.34's, 80.864'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 45.553'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3127.132'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 294.341'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33846.029'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.864'MB/sec zip pipeline throughput = 80.864'MB/sec entries = 922592 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.48 avg-zval = 35.77 usrkeys = 922592 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.65 avg-zval = 35.77 TagRS{ kinds = RS_Key0_TagN bytes = 144240 } TagArray{ size = 549749 bytes = 1855424 } seq expand size = 3848243 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.3772 value = 3.2842 dict = 1.00 all = 3.5439 } Zip/UnZip{ index = 0.0964 value = 0.3045 dict = 1.00 all = 0.2822 } ---------------------------- total value len = 0.676014 GB avg = 0.119 KB (by entry num) total key len = 0.159432 GB avg = 0.028 KB total ukey len = 0.113880 GB avg = 0.020 KB total ukey num = 0.005694000 Billion total entry num = 0.005694000 Billion write speed all = 0.030710884 MB/sec (with seq num) write speed all = 0.029036399 MB/sec (without seq num) 2024/11/22-16:25:46.715675 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214406: 922592 keys, 37872795 bytes, temperature: kWarm 2024/11/22-16:25:46.715736 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263946715704, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214406, "file_size": 37872795, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3376771289, "table_properties": {"data_size": 33002224, "index_size": 1778112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25832576, "raw_average_key_size": 28, "raw_value_size": 108385288, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922592, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263943, "oldest_key_time": 0, "file_creation_time": 1732263943, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214406, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:25:46.750022 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214407.sst) /dev/shm/ToplingTemp/Topling-197490-pygXys 2024/11/22-16:25:47.559155 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214407.sst): first pass time = 0.81's, 165.886'MB/sec 2024/11/22-16:25:48.897500 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214407.sst): old prealloc_size = 147639500, real_size = 34199344 2024/11/22-16:25:48.898056 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214407.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922909 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0322 GB avg-val = 117.43 avg-zval = 34.89 2024/11/22-16:25:48.898797 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214407.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:25:49.801735 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214407.sst): second pass time = 1.34's, 81.017'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.216'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3313.966'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 297.615'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34292.827'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.017'MB/sec zip pipeline throughput = 81.017'MB/sec entries = 922909 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.43 avg-zval = 34.89 usrkeys = 922909 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.43 avg-zval = 34.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922909 bytes = 3114832 } seq expand size = 6460363 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4458 value = 3.3656 dict = 1.00 all = 3.5161 } Zip/UnZip{ index = 0.0957 value = 0.2971 dict = 1.00 all = 0.2844 } ---------------------------- total value len = 0.790851 GB avg = 0.120 KB (by entry num) total key len = 0.185273 GB avg = 0.028 KB total ukey len = 0.132338 GB avg = 0.020 KB total ukey num = 0.006616909 Billion total entry num = 0.006616909 Billion write speed all = 0.035878081 MB/sec (with seq num) write speed all = 0.033932411 MB/sec (without seq num) 2024/11/22-16:25:49.837452 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214407: 922909 keys, 38172092 bytes, temperature: kWarm 2024/11/22-16:25:49.837539 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263949837483, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214407, "file_size": 38172092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701073, "largest_seqno": 3376772374, "table_properties": {"data_size": 32201376, "index_size": 1767040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25841452, "raw_average_key_size": 28, "raw_value_size": 108376418, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922909, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263946, "oldest_key_time": 0, "file_creation_time": 1732263946, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214407, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:25:49.872989 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214408.sst) /dev/shm/ToplingTemp/Topling-197490-9zYUDu 2024/11/22-16:25:50.744880 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214408.sst): first pass time = 0.87's, 153.945'MB/sec 2024/11/22-16:25:52.075817 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214408.sst): old prealloc_size = 147639500, real_size = 33833616 2024/11/22-16:25:52.076363 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214408.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921736 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.61 avg-zval = 34.54 2024/11/22-16:25:52.077107 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214408.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:25:52.999966 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214408.sst): second pass time = 1.33's, 81.489'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.907'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.308'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 294.793'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 36275.366'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.489'MB/sec zip pipeline throughput = 81.489'MB/sec entries = 921736 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.61 avg-zval = 34.54 usrkeys = 921736 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.61 avg-zval = 34.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921736 bytes = 3110880 } seq expand size = 6452152 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4431 value = 3.4051 dict = 1.00 all = 3.5504 } Zip/UnZip{ index = 0.0958 value = 0.2937 dict = 1.00 all = 0.2817 } ---------------------------- total value len = 0.905712 GB avg = 0.120 KB (by entry num) total key len = 0.211082 GB avg = 0.028 KB total ukey len = 0.150773 GB avg = 0.020 KB total ukey num = 0.007538645 Billion total entry num = 0.007538645 Billion write speed all = 0.041043670 MB/sec (with seq num) write speed all = 0.038827229 MB/sec (without seq num) 2024/11/22-16:25:53.033370 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214408: 921736 keys, 37803542 bytes, temperature: kWarm 2024/11/22-16:25:53.033433 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263953033402, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214408, "file_size": 37803542, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3376771337, "table_properties": {"data_size": 31837760, "index_size": 1765248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25808608, "raw_average_key_size": 28, "raw_value_size": 108409168, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921736, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263949, "oldest_key_time": 0, "file_creation_time": 1732263949, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214408, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:25:53.069031 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214409.sst) /dev/shm/ToplingTemp/Topling-197490-czIvjt 2024/11/22-16:25:53.895912 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214409.sst): first pass time = 0.83's, 162.326'MB/sec 2024/11/22-16:25:55.240015 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214409.sst): old prealloc_size = 147639500, real_size = 35897008 2024/11/22-16:25:55.240583 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214409.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924931 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0339 GB avg-val = 117.11 avg-zval = 36.63 2024/11/22-16:25:55.241328 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214409.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:25:56.107051 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214409.sst): second pass time = 1.34's, 80.625'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 45.990'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3248.739'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.13's, 310.832'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33274.378'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.625'MB/sec zip pipeline throughput = 80.625'MB/sec entries = 924931 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.11 avg-zval = 36.63 usrkeys = 924931 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.11 avg-zval = 36.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924931 bytes = 3121664 } seq expand size = 6474517 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.3751 value = 3.1969 dict = 1.00 all = 3.3660 } Zip/UnZip{ index = 0.0964 value = 0.3128 dict = 1.00 all = 0.2971 } ---------------------------- total value len = 1.020506 GB avg = 0.121 KB (by entry num) total key len = 0.236980 GB avg = 0.028 KB total ukey len = 0.169272 GB avg = 0.020 KB total ukey num = 0.008463576 Billion total entry num = 0.008463576 Billion write speed all = 0.046209021 MB/sec (with seq num) write speed all = 0.043720924 MB/sec (without seq num) 2024/11/22-16:25:56.143394 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214409: 924931 keys, 39874250 bytes, temperature: kWarm 2024/11/22-16:25:56.143453 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263956143421, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214409, "file_size": 39874250, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701122, "largest_seqno": 3376771679, "table_properties": {"data_size": 33882592, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25898068, "raw_average_key_size": 28, "raw_value_size": 108319686, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924931, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263953, "oldest_key_time": 0, "file_creation_time": 1732263953, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214409, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:25:56.181849 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214410.sst) /dev/shm/ToplingTemp/Topling-197490-pXLDKq 2024/11/22-16:25:56.965126 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214410.sst): first pass time = 0.78's, 171.361'MB/sec 2024/11/22-16:25:58.287364 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214410.sst): old prealloc_size = 147639500, real_size = 33764016 2024/11/22-16:25:58.287959 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214410.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923135 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.39 avg-zval = 34.42 2024/11/22-16:25:58.288696 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214410.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:25:59.209399 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214410.sst): second pass time = 1.32's, 81.999'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.929'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3063.501'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.291'MB/sec write SST data time = 0.13's, 292.117'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34476.630'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.999'MB/sec zip pipeline throughput = 81.999'MB/sec entries = 923135 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 34.42 usrkeys = 923135 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 34.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923135 bytes = 3115600 } seq expand size = 6461945 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5146 value = 3.4103 dict = 1.00 all = 3.5566 } Zip/UnZip{ index = 0.0951 value = 0.2932 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 1.135338 GB avg = 0.121 KB (by entry num) total key len = 0.262828 GB avg = 0.028 KB total ukey len = 0.187734 GB avg = 0.020 KB total ukey num = 0.009386711 Billion total entry num = 0.009386711 Billion write speed all = 0.051372741 MB/sec (with seq num) write speed all = 0.048613578 MB/sec (without seq num) 2024/11/22-16:25:59.241545 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214410: 923135 keys, 37737514 bytes, temperature: kWarm 2024/11/22-16:25:59.241610 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263959241579, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214410, "file_size": 37737514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701184, "largest_seqno": 3376771870, "table_properties": {"data_size": 31777120, "index_size": 1755904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847780, "raw_average_key_size": 28, "raw_value_size": 108370006, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923135, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263956, "oldest_key_time": 0, "file_creation_time": 1732263956, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214410, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:25:59.277295 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214411.sst) /dev/shm/ToplingTemp/Topling-197490-3Pjsyr 2024/11/22-16:25:59.578242 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214411.sst): first pass time = 0.30's, 251.557'MB/sec 2024/11/22-16:26:00.319272 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214411.sst): old prealloc_size = 147639500, real_size = 19223024 2024/11/22-16:26:00.319637 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214411.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 518480 prefix = 4 raw-key = 0.0083 GB zip-key = 0.0010 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0612 GB zip-val = 0.0181 GB avg-val = 118.00 avg-zval = 34.91 2024/11/22-16:26:00.320078 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214411.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.076 GB 2024/11/22-16:26:00.872046 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00/214411.sst): second pass time = 0.74's, 82.623'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.55's, 38.820'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2924.953'MB/sec (index lex order gen) rebuild zvType time = 0.48's, 0.270'MB/sec write SST data time = 0.07's, 296.963'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.612'MB, throughput = 18568.310'MB/sec zip my value time = 0.74's, unzip length = 0.061'GB zip my value throughput = 82.623'MB/sec zip pipeline throughput = 82.623'MB/sec entries = 518480 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 34.91 usrkeys = 518480 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 34.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 518480 bytes = 1749888 } seq expand size = 3629360 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0104 GB value = 0.0612 GB dict = 0.61 MB all = 0.0757 GB } __ZipSize{ index = 0.0010 GB value = 0.0181 GB dict = 0.61 MB all = 0.0215 GB } UnZip/Zip{ index = 10.4377 value = 3.3802 dict = 1.00 all = 3.5277 } Zip/UnZip{ index = 0.0958 value = 0.2958 dict = 1.00 all = 0.2835 } ---------------------------- total value len = 1.200148 GB avg = 0.121 KB (by entry num) total key len = 0.277345 GB avg = 0.028 KB total ukey len = 0.198104 GB avg = 0.020 KB total ukey num = 0.009905191 Billion total entry num = 0.009905191 Billion write speed all = 0.054284148 MB/sec (with seq num) write speed all = 0.051372759 MB/sec (without seq num) 2024/11/22-16:26:00.894598 140025205032512 [compaction_job.cc:1948] [default] [JOB 4473] Generated table #214411: 518480 keys, 21458019 bytes, temperature: kWarm 2024/11/22-16:26:00.894655 140025205032512 EVENT_LOG_v1 {"time_micros": 1732263960894624, "cf_name": "default", "job": 4473, "event": "table_file_creation", "file_number": 214411, "file_size": 21458019, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3277343121, "largest_seqno": 3376073270, "table_properties": {"data_size": 18099744, "index_size": 993472, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 14517440, "raw_average_key_size": 28, "raw_value_size": 61180640, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 518480, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263959, "oldest_key_time": 0, "file_creation_time": 1732263959, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 214411, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:26:00.916445 140025205032512 [compaction_job.cc:703] job-04473: subcompact[0], size: 0.401595 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04473/att-00 [214401,214402,214403,214404,214405,214406,214407,214408,214409,214410,214411] 2024/11/22-16:26:00.983285 140025205032512 (Original Log Time 2024/11/22-16:25:27.108006) EVENT_LOG_v1 {"time_micros": 1732263927107967, "job": 4473, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [214357], "files_L3": [214397, 214398, 214308, 214309, 214279, 214280, 214281, 214282, 214285, 214286, 214287, 214376], "score": 0.947878, "input_data_size": 425503964, "oldest_snapshot_seqno": 3377457668} 2024/11/22-16:26:00.983289 140025205032512 (Original Log Time 2024/11/22-16:26:00.982674) [compaction_job.cc:2022] [default] [JOB 4473] Compacted 1@2 + 12@3 files to L3 => 401594561 bytes 2024/11/22-16:26:00.983290 140025205032512 (Original Log Time 2024/11/22-16:26:00.983228) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 12.6 rd, 11.9 wr, level 3, files in(1, 12) out(11 +0 blob) MB in(27.2, 378.6 +0.0 blob) out(383.0 +0.0 blob), read-write-amplify(29.0) write-amplify(14.1) OK, records in: 10002779, records dropped: 97588 output_compression: Snappy 2024/11/22-16:26:00.983296 140025205032512 (Original Log Time 2024/11/22-16:26:00.983269) EVENT_LOG_v1 {"time_micros": 1732263960983250, "cf": "default", "job": 4473, "event": "compaction_finished", "compaction_time_micros": 33808446, "compaction_time_cpu_micros": 15286213, "output_level": 3, "num_output_files": 11, "total_output_size": 401594561, "num_input_records": 10002779, "num_output_records": 9905191, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 251602305, "file_range_sync_nanos": 0, "file_fsync_nanos": 10516, "file_prepare_write_nanos": 24841, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/22-16:26:01.092930 140025205032512 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732263926,"executesMs":34009,"compactionJobId":4473,"attempt":0,"compactionInputRawBytes":1431733751,"compactionInputZipBytes":425503964,"compactionOutputRawBytes":0,"compactionOutputZipBytes":401594561,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-6e923cd2058a0ff74dcd70e83d027082-1383d991fe7bb9f7-00"}}, response = 2024/11/22-16:26:01.092988 140025205032512 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04473/att-00: olev 3, work 34.010 s, result 4.885 ms, install 0.638 ms, input{raw 1.333 GiB zip 405.792 MiB}