2024/11/22-15:39:12.321791 140024718255680 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00: fork to child time = 0.019236 sec 2024/11/22-15:39:12.322558 140024718255680 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00 2024/11/22-15:39:12.329810 140024718255680 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00 2024/11/22-15:39:12.331400 140024718255680 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-15:39:12.331622 140024718255680 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-15:39:12.385670 140024718255680 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-03214/att-00/MANIFEST-206347 2024/11/22-15:39:12.398762 140024718255680 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-03214/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 211956, last_sequence is 3342338979, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 211779 2024/11/22-15:39:12.398767 140024718255680 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-15:39:12.398996 140024718255680 [version_set.cc:5721] Creating manifest 211956 2024/11/22-15:39:12.430099 140024718255680 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-15:39:12.450728 140024718255680 [compaction_job.cc:2437] [default] [JOB 3214] Compacting 1@2 + 10@3 files to L3, score 0.86, subcompactions 1 : 1 2024/11/22-15:39:12.450742 140024718255680 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 2574 Base level 2, inputs: [211926(22MB)], [211862(37MB) 211863(37MB) 211864(37MB) 211865(37MB) 211866(38MB) 211867(38MB) 211868(36MB) 211869(35MB) 211870(36MB) 211946(37MB)] 2024/11/22-15:39:12.456164 140024718255680 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-15:39:12.456190 140024718255680 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-15:39:12.456206 140024718255680 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-15:39:12.456211 140024718255680 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-15:39:12.456221 140024718255680 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-15:39:12.881999 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211957.sst) /dev/shm/ToplingTemp/Topling-180049-kKDVJF 2024/11/22-15:39:13.551611 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211957.sst): first pass time = 0.67's, 200.452'MB/sec 2024/11/22-15:39:15.442754 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211957.sst): old prealloc_size = 147639500, real_size = 35214432 2024/11/22-15:39:15.443303 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211957.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921946 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.58 avg-zval = 36.03 2024/11/22-15:39:15.444073 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211957.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:39:16.702360 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211957.sst): second pass time = 1.89's, 57.344'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.26's, 31.109'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3397.681'MB/sec (index lex order gen) rebuild zvType time = 1.13's, 0.203'MB/sec write SST data time = 0.13's, 310.516'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 38337.179'MB/sec zip my value time = 1.89's, unzip length = 0.108'GB zip my value throughput = 57.344'MB/sec zip pipeline throughput = 57.344'MB/sec entries = 921946 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.58 avg-zval = 36.03 usrkeys = 921946 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.58 avg-zval = 36.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921946 bytes = 3111584 } seq expand size = 6453622 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4432 value = 3.2634 dict = 1.00 all = 3.4253 } Zip/UnZip{ index = 0.0958 value = 0.3064 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 0.114857 GB avg = 0.125 KB (by entry num) total key len = 0.025814 GB avg = 0.028 KB total ukey len = 0.018439 GB avg = 0.020 KB total ukey num = 0.000921946 Billion total entry num = 0.000921946 Billion write speed all = 0.005762007 MB/sec (with seq num) write speed all = 0.005459898 MB/sec (without seq num) 2024/11/22-15:39:16.736639 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211957: 921946 keys, 39184660 bytes, temperature: kWarm 2024/11/22-15:39:16.736724 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261156736674, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211957, "file_size": 39184660, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702550, "largest_seqno": 3341907894, "table_properties": {"data_size": 33218128, "index_size": 1765632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25814488, "raw_average_key_size": 28, "raw_value_size": 108403256, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921946, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261152, "oldest_key_time": 0, "file_creation_time": 1732261152, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211957, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:16.776805 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211958.sst) /dev/shm/ToplingTemp/Topling-180049-UnVcfG 2024/11/22-15:39:17.839545 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211958.sst): first pass time = 1.06's, 126.298'MB/sec 2024/11/22-15:39:19.799999 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211958.sst): old prealloc_size = 147639500, real_size = 35277744 2024/11/22-15:39:19.800631 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211958.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923127 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.39 avg-zval = 36.06 2024/11/22-15:39:19.801402 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211958.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:39:20.783914 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211958.sst): second pass time = 1.96's, 55.297'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 39.893'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2851.904'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.16's, 252.560'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32203.272'MB/sec zip my value time = 1.96's, unzip length = 0.108'GB zip my value throughput = 55.297'MB/sec zip pipeline throughput = 55.782'MB/sec entries = 923127 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 36.06 usrkeys = 923127 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 36.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923127 bytes = 3115568 } seq expand size = 6461889 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5123 value = 3.2553 dict = 1.00 all = 3.4195 } Zip/UnZip{ index = 0.0951 value = 0.3072 dict = 1.00 all = 0.2924 } ---------------------------- total value len = 0.229689 GB avg = 0.124 KB (by entry num) total key len = 0.051662 GB avg = 0.028 KB total ukey len = 0.036901 GB avg = 0.020 KB total ukey num = 0.001845073 Billion total entry num = 0.001845073 Billion write speed all = 0.011522425 MB/sec (with seq num) write speed all = 0.010917922 MB/sec (without seq num) 2024/11/22-15:39:20.818597 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211958: 923127 keys, 39251010 bytes, temperature: kWarm 2024/11/22-15:39:20.818661 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261160818629, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211958, "file_size": 39251010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702575, "largest_seqno": 3341907668, "table_properties": {"data_size": 33290464, "index_size": 1756288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847556, "raw_average_key_size": 28, "raw_value_size": 108370174, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923127, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261156, "oldest_key_time": 0, "file_creation_time": 1732261156, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211958, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:20.857995 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211959.sst) /dev/shm/ToplingTemp/Topling-180049-YdWSIE 2024/11/22-15:39:21.915463 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211959.sst): first pass time = 1.06's, 126.929'MB/sec 2024/11/22-15:39:23.734556 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211959.sst): old prealloc_size = 147639500, real_size = 35137456 2024/11/22-15:39:23.735145 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211959.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921848 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.60 avg-zval = 35.96 2024/11/22-15:39:23.737526 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211959.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:39:24.751812 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211959.sst): second pass time = 1.82's, 59.632'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 38.446'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3085.862'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.294'MB/sec write SST data time = 0.23's, 168.002'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34648.885'MB/sec zip my value time = 1.82's, unzip length = 0.108'GB zip my value throughput = 59.632'MB/sec zip pipeline throughput = 91.262'MB/sec entries = 921848 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.60 avg-zval = 35.96 usrkeys = 921848 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.60 avg-zval = 35.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921848 bytes = 3111248 } seq expand size = 6452936 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5107 value = 3.2699 dict = 1.00 all = 3.4320 } Zip/UnZip{ index = 0.0951 value = 0.3058 dict = 1.00 all = 0.2914 } ---------------------------- total value len = 0.344548 GB avg = 0.125 KB (by entry num) total key len = 0.077474 GB avg = 0.028 KB total ukey len = 0.055338 GB avg = 0.020 KB total ukey num = 0.002766921 Billion total entry num = 0.002766921 Billion write speed all = 0.017280633 MB/sec (with seq num) write speed all = 0.016374251 MB/sec (without seq num) 2024/11/22-15:39:24.788747 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211959: 921848 keys, 39107516 bytes, temperature: kWarm 2024/11/22-15:39:24.788831 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261164788788, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211959, "file_size": 39107516, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702617, "largest_seqno": 3341907978, "table_properties": {"data_size": 33152672, "index_size": 1754112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25811744, "raw_average_key_size": 28, "raw_value_size": 108406012, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921848, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261160, "oldest_key_time": 0, "file_creation_time": 1732261160, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211959, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:24.852488 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211960.sst) /dev/shm/ToplingTemp/Topling-180049-2AD7CD 2024/11/22-15:39:25.826629 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211960.sst): first pass time = 0.97's, 137.787'MB/sec 2024/11/22-15:39:27.468513 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211960.sst): old prealloc_size = 147639500, real_size = 35151472 2024/11/22-15:39:27.469055 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211960.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923170 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.39 avg-zval = 35.92 2024/11/22-15:39:27.469759 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211960.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:39:28.362772 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211960.sst): second pass time = 1.64's, 66.049'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.756'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3345.598'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.13's, 308.216'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34407.526'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 66.049'MB/sec zip pipeline throughput = 78.719'MB/sec entries = 923170 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 35.92 usrkeys = 923170 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 35.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923170 bytes = 3115712 } seq expand size = 6462190 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5124 value = 3.2677 dict = 1.00 all = 3.4302 } Zip/UnZip{ index = 0.0951 value = 0.3060 dict = 1.00 all = 0.2915 } ---------------------------- total value len = 0.459379 GB avg = 0.124 KB (by entry num) total key len = 0.103323 GB avg = 0.028 KB total ukey len = 0.073802 GB avg = 0.020 KB total ukey num = 0.003690091 Billion total entry num = 0.003690091 Billion write speed all = 0.023037687 MB/sec (with seq num) write speed all = 0.021829073 MB/sec (without seq num) 2024/11/22-15:39:28.400029 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211960: 923170 keys, 39128596 bytes, temperature: kWarm 2024/11/22-15:39:28.400093 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261168400063, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211960, "file_size": 39128596, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3341907309, "table_properties": {"data_size": 33164128, "index_size": 1756352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25848760, "raw_average_key_size": 28, "raw_value_size": 108369020, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923170, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261164, "oldest_key_time": 0, "file_creation_time": 1732261164, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211960, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:28.441276 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211961.sst) /dev/shm/ToplingTemp/Topling-180049-Yfb2RG 2024/11/22-15:39:29.831803 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211961.sst): first pass time = 1.39's, 96.525'MB/sec 2024/11/22-15:39:31.489793 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211961.sst): old prealloc_size = 147639500, real_size = 36624912 2024/11/22-15:39:31.490365 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211961.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922520 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0346 GB avg-val = 117.49 avg-zval = 37.55 2024/11/22-15:39:31.491120 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211961.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:39:32.337631 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211961.sst): second pass time = 1.66's, 65.398'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 47.747'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.048'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 317.860'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35119.598'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.398'MB/sec zip pipeline throughput = 93.037'MB/sec entries = 922520 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.49 avg-zval = 37.55 usrkeys = 922520 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.49 avg-zval = 37.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922520 bytes = 2998208 } seq expand size = 6457640 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.5119 value = 3.1291 dict = 1.00 all = 3.3155 } Zip/UnZip{ index = 0.0951 value = 0.3196 dict = 1.00 all = 0.3016 } ---------------------------- total value len = 0.574224 GB avg = 0.124 KB (by entry num) total key len = 0.129153 GB avg = 0.028 KB total ukey len = 0.092252 GB avg = 0.020 KB total ukey num = 0.004612611 Billion total entry num = 0.004612611 Billion write speed all = 0.028792425 MB/sec (with seq num) write speed all = 0.027281904 MB/sec (without seq num) 2024/11/22-15:39:32.374405 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211961: 922520 keys, 40481616 bytes, temperature: kWarm 2024/11/22-15:39:32.374468 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261172374438, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211961, "file_size": 40481616, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868619, "largest_seqno": 3273700579, "table_properties": {"data_size": 34638912, "index_size": 1755200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25830560, "raw_average_key_size": 28, "raw_value_size": 108387260, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922520, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261168, "oldest_key_time": 0, "file_creation_time": 1732261168, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211961, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:32.411702 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211962.sst) /dev/shm/ToplingTemp/Topling-180049-DFUMYF 2024/11/22-15:39:33.270388 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211962.sst): first pass time = 0.86's, 156.313'MB/sec 2024/11/22-15:39:34.728445 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211962.sst): old prealloc_size = 147639500, real_size = 36568448 2024/11/22-15:39:34.729005 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211962.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922651 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0346 GB avg-val = 117.47 avg-zval = 37.47 2024/11/22-15:39:34.729692 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211962.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:39:35.559978 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211962.sst): second pass time = 1.46's, 74.370'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.514'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3294.092'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 304.691'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33366.401'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.370'MB/sec zip pipeline throughput = 74.370'MB/sec entries = 922651 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.47 avg-zval = 37.47 usrkeys = 922651 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.93 avg-zval = 37.47 TagRS{ kinds = RS_Key0_TagN bytes = 144280 } TagArray{ size = 851968 bytes = 2768912 } seq expand size = 5963776 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0403 GB } UnZip/Zip{ index = 10.4455 value = 3.1351 dict = 1.00 all = 3.3272 } Zip/UnZip{ index = 0.0957 value = 0.3190 dict = 1.00 all = 0.3006 } ---------------------------- total value len = 0.688571 GB avg = 0.124 KB (by entry num) total key len = 0.154987 GB avg = 0.028 KB total ukey len = 0.110705 GB avg = 0.020 KB total ukey num = 0.005535262 Billion total entry num = 0.005535262 Billion write speed all = 0.034526142 MB/sec (with seq num) write speed all = 0.032713714 MB/sec (without seq num) 2024/11/22-15:39:35.592029 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211962: 922651 keys, 40339824 bytes, temperature: kWarm 2024/11/22-15:39:35.592087 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261175592056, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211962, "file_size": 40339824, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700489, "table_properties": {"data_size": 34570992, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25834228, "raw_average_key_size": 28, "raw_value_size": 108383610, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922651, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261172, "oldest_key_time": 0, "file_creation_time": 1732261172, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211962, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:35.630075 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211963.sst) /dev/shm/ToplingTemp/Topling-180049-SUBOOD 2024/11/22-15:39:36.500111 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211963.sst): first pass time = 0.87's, 154.272'MB/sec 2024/11/22-15:39:37.995386 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211963.sst): old prealloc_size = 147639500, real_size = 36684048 2024/11/22-15:39:37.995939 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211963.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925156 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0347 GB avg-val = 117.08 avg-zval = 37.51 2024/11/22-15:39:37.996321 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211963.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:39:38.727104 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211963.sst): second pass time = 1.49's, 72.468'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 53.213'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3286.079'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.384'MB/sec write SST data time = 0.13's, 302.858'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35369.089'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.468'MB/sec zip pipeline throughput = 72.468'MB/sec entries = 925156 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.08 avg-zval = 37.51 usrkeys = 925156 avg-key = 20.00 avg-zkey = 1.89 avg-val = 119.98 avg-zval = 37.51 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 384073 bytes = 1248256 } seq expand size = 2688511 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0347 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.5832 value = 3.1210 dict = 1.00 all = 3.4471 } Zip/UnZip{ index = 0.0945 value = 0.3204 dict = 1.00 all = 0.2901 } ---------------------------- total value len = 0.799573 GB avg = 0.124 KB (by entry num) total key len = 0.180892 GB avg = 0.028 KB total ukey len = 0.129208 GB avg = 0.020 KB total ukey num = 0.006460418 Billion total entry num = 0.006460418 Billion write speed all = 0.040124399 MB/sec (with seq num) write speed all = 0.038009318 MB/sec (without seq num) 2024/11/22-15:39:38.761944 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211963: 925156 keys, 38936027 bytes, temperature: kWarm 2024/11/22-15:39:38.762005 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261178761973, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211963, "file_size": 38936027, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 34704192, "index_size": 1748352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904368, "raw_average_key_size": 28, "raw_value_size": 108313420, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925156, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261175, "oldest_key_time": 0, "file_creation_time": 1732261175, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211963, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:38.797349 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211964.sst) /dev/shm/ToplingTemp/Topling-180049-nJW81F 2024/11/22-15:39:39.520343 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211964.sst): first pass time = 0.72's, 185.649'MB/sec 2024/11/22-15:39:41.091759 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211964.sst): old prealloc_size = 147639500, real_size = 36451312 2024/11/22-15:39:41.092334 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211964.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926902 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.80 avg-zval = 37.17 2024/11/22-15:39:41.092374 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211964.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:39:41.742387 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211964.sst): second pass time = 1.57's, 68.946'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 57.345'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3246.361'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.443'MB/sec write SST data time = 0.13's, 293.234'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32865.703'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.946'MB/sec zip pipeline throughput = 68.946'MB/sec entries = 926902 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 37.17 usrkeys = 926902 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 37.17 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5123 value = 3.1421 dict = 1.00 all = 3.5974 } Zip/UnZip{ index = 0.0951 value = 0.3183 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 0.907838 GB avg = 0.123 KB (by entry num) total key len = 0.206845 GB avg = 0.028 KB total ukey len = 0.147746 GB avg = 0.020 KB total ukey num = 0.007387320 Billion total entry num = 0.007387320 Billion write speed all = 0.045611481 MB/sec (with seq num) write speed all = 0.043193240 MB/sec (without seq num) 2024/11/22-15:39:41.775885 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211964: 926902 keys, 37309582 bytes, temperature: kWarm 2024/11/22-15:39:41.775950 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261181775916, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211964, "file_size": 37309582, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34455968, "index_size": 1763456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953256, "raw_average_key_size": 28, "raw_value_size": 108264588, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926902, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261178, "oldest_key_time": 0, "file_creation_time": 1732261178, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211964, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:41.815123 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211965.sst) /dev/shm/ToplingTemp/Topling-180049-XJDIcG 2024/11/22-15:39:42.641177 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211965.sst): first pass time = 0.83's, 162.489'MB/sec 2024/11/22-15:39:44.205913 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211965.sst): old prealloc_size = 147639500, real_size = 36880704 2024/11/22-15:39:44.206439 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211965.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925485 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 117.02 avg-zval = 37.70 2024/11/22-15:39:44.206711 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211965.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:39:44.897474 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211965.sst): second pass time = 1.56's, 69.267'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 55.698'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3493.601'MB/sec (index lex order gen) rebuild zvType time = 0.56's, 0.411'MB/sec write SST data time = 0.13's, 299.482'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32833.161'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.267'MB/sec zip pipeline throughput = 69.267'MB/sec entries = 925485 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.02 avg-zval = 37.70 usrkeys = 925485 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.50 avg-zval = 37.70 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 194909 bytes = 633472 } seq expand size = 1364363 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5123 value = 3.1043 dict = 1.00 all = 3.4845 } Zip/UnZip{ index = 0.0951 value = 0.3221 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 1.017507 GB avg = 0.122 KB (by entry num) total key len = 0.232759 GB avg = 0.028 KB total ukey len = 0.166256 GB avg = 0.020 KB total ukey num = 0.008312805 Billion total entry num = 0.008312805 Billion write speed all = 0.051152738 MB/sec (with seq num) write speed all = 0.048431890 MB/sec (without seq num) 2024/11/22-15:39:44.930289 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211965: 925485 keys, 38518001 bytes, temperature: kWarm 2024/11/22-15:39:44.930348 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261184930316, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211965, "file_size": 38518001, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3313774006, "table_properties": {"data_size": 34888368, "index_size": 1760768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25913580, "raw_average_key_size": 28, "raw_value_size": 108304266, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925485, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261181, "oldest_key_time": 0, "file_creation_time": 1732261181, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211965, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:44.967565 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211966.sst) /dev/shm/ToplingTemp/Topling-180049-wd4NXC 2024/11/22-15:39:45.776054 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211966.sst): first pass time = 0.81's, 166.017'MB/sec 2024/11/22-15:39:47.090253 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211966.sst): old prealloc_size = 147639500, real_size = 35171664 2024/11/22-15:39:47.090818 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211966.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922473 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.50 avg-zval = 35.95 2024/11/22-15:39:47.091562 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211966.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:39:47.981900 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211966.sst): second pass time = 1.31's, 82.513'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.901'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3280.982'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.302'MB/sec write SST data time = 0.13's, 304.435'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 29916.315'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.513'MB/sec zip pipeline throughput = 82.513'MB/sec entries = 922473 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.50 avg-zval = 35.95 usrkeys = 922473 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.50 avg-zval = 35.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922473 bytes = 3113360 } seq expand size = 6457311 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.3759 value = 3.2684 dict = 1.00 all = 3.4289 } Zip/UnZip{ index = 0.0964 value = 0.3060 dict = 1.00 all = 0.2916 } ---------------------------- total value len = 1.132352 GB avg = 0.123 KB (by entry num) total key len = 0.258588 GB avg = 0.028 KB total ukey len = 0.184706 GB avg = 0.020 KB total ukey num = 0.009235278 Billion total entry num = 0.009235278 Billion write speed all = 0.056901072 MB/sec (with seq num) write speed all = 0.053878672 MB/sec (without seq num) 2024/11/22-15:39:48.013962 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211966: 922473 keys, 39143484 bytes, temperature: kWarm 2024/11/22-15:39:48.014022 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261188013991, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211966, "file_size": 39143484, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701362, "largest_seqno": 3341908166, "table_properties": {"data_size": 33162752, "index_size": 1778112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25829244, "raw_average_key_size": 28, "raw_value_size": 108388578, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922473, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261184, "oldest_key_time": 0, "file_creation_time": 1732261184, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211966, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:48.048334 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211967.sst) /dev/shm/ToplingTemp/Topling-180049-yl3kNF 2024/11/22-15:39:48.107894 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211967.sst): first pass time = 0.06's, 238.215'MB/sec 2024/11/22-15:39:48.227480 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211967.sst): old prealloc_size = 147639500, real_size = 4386672 2024/11/22-15:39:48.227581 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211967.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 97834 prefix = 4 raw-key = 0.0016 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0114 GB zip-val = 0.0042 GB avg-val = 116.94 avg-zval = 42.67 2024/11/22-15:39:48.227696 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211967.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.014 GB 2024/11/22-15:39:48.307521 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00/211967.sst): second pass time = 0.12's, 96.040'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.08's, 60.128'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2331.887'MB/sec (index lex order gen) rebuild zvType time = 0.07's, 0.367'MB/sec write SST data time = 0.01's, 359.337'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.118'MB, throughput = 5164.771'MB/sec zip my value time = 0.12's, unzip length = 0.011'GB zip my value throughput = 96.040'MB/sec zip pipeline throughput = 96.040'MB/sec entries = 97834 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.94 avg-zval = 42.67 usrkeys = 97834 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.94 avg-zval = 42.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 97834 bytes = 330208 } seq expand size = 684838 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0020 GB value = 0.0114 GB dict = 0.12 MB all = 0.0142 GB } __ZipSize{ index = 0.0002 GB value = 0.0042 GB dict = 0.12 MB all = 0.0048 GB } UnZip/Zip{ index = 10.4310 value = 2.7408 dict = 1.00 all = 2.9468 } Zip/UnZip{ index = 0.0959 value = 0.3649 dict = 1.00 all = 0.3394 } ---------------------------- total value len = 1.144478 GB avg = 0.123 KB (by entry num) total key len = 0.261327 GB avg = 0.028 KB total ukey len = 0.186662 GB avg = 0.020 KB total ukey num = 0.009333112 Billion total entry num = 0.009333112 Billion write speed all = 0.057508419 MB/sec (with seq num) write speed all = 0.054454042 MB/sec (without seq num) 2024/11/22-15:39:48.316707 140024718255680 [compaction_job.cc:1948] [default] [JOB 3214] Generated table #211967: 97834 keys, 4812141 bytes, temperature: kWarm 2024/11/22-15:39:48.316756 140024718255680 EVENT_LOG_v1 {"time_micros": 1732261188316728, "cf_name": "default", "job": 3214, "event": "table_file_creation", "file_number": 211967, "file_size": 4812141, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3341907357, "table_properties": {"data_size": 4174432, "index_size": 187584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2739352, "raw_average_key_size": 28, "raw_value_size": 11441076, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 97834, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261188, "oldest_key_time": 0, "file_creation_time": 1732261188, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 211967, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:39:48.325874 140024718255680 [compaction_job.cc:703] job-03214: subcompact[0], size: 0.396212 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03214/att-00 [211957,211958,211959,211960,211961,211962,211963,211964,211965,211966,211967] 2024/11/22-15:39:48.400247 140024718255680 (Original Log Time 2024/11/22-15:39:12.450787) EVENT_LOG_v1 {"time_micros": 1732261152450753, "job": 3214, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [211926], "files_L3": [211862, 211863, 211864, 211865, 211866, 211867, 211868, 211869, 211870, 211946], "score": 0.863191, "input_data_size": 416114856, "oldest_snapshot_seqno": 3342338979} 2024/11/22-15:39:48.400250 140024718255680 (Original Log Time 2024/11/22-15:39:48.399635) [compaction_job.cc:2022] [default] [JOB 3214] Compacted 1@2 + 10@3 files to L3 => 396212457 bytes 2024/11/22-15:39:48.400251 140024718255680 (Original Log Time 2024/11/22-15:39:48.400186) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 11.6 rd, 11.0 wr, level 3, files in(1, 10) out(11 +0 blob) MB in(22.7, 374.2 +0.0 blob) out(377.9 +0.0 blob), read-write-amplify(34.2) write-amplify(16.7) OK, records in: 9430427, records dropped: 97315 output_compression: Snappy 2024/11/22-15:39:48.400258 140024718255680 (Original Log Time 2024/11/22-15:39:48.400231) EVENT_LOG_v1 {"time_micros": 1732261188400213, "cf": "default", "job": 3214, "event": "compaction_finished", "compaction_time_micros": 35875092, "compaction_time_cpu_micros": 15139739, "output_level": 3, "num_output_files": 11, "total_output_size": 396212457, "num_input_records": 9430427, "num_output_records": 9333112, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 259492751, "file_range_sync_nanos": 0, "file_fsync_nanos": 10950, "file_prepare_write_nanos": 23417, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/22-15:39:48.535382 140024718255680 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732261152,"executesMs":36071,"compactionJobId":3214,"attempt":0,"compactionInputRawBytes":1370177126,"compactionInputZipBytes":416114856,"compactionOutputRawBytes":0,"compactionOutputZipBytes":396212457,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-f92e53e444cb0dae9c928bacd6a376f6-5387b67f45d504bc-00"}}, response = 2024/11/22-15:39:48.535437 140024718255680 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-03214/att-00: olev 3, work 36.071 s, result 6.265 ms, install 0.648 ms, input{raw 1.276 GiB zip 396.838 MiB}