2024/11/24-00:16:16.072326 139957248734784 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00: fork to child time = 0.007851 sec 2024/11/24-00:16:16.073090 139957248734784 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00 2024/11/24-00:16:16.080268 139957248734784 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00 2024/11/24-00:16:16.081165 139957248734784 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-00:16:16.081498 139957248734784 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-00:16:16.129143 139957248734784 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38441/att-00/MANIFEST-206347 2024/11/24-00:16:16.153135 139957248734784 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38441/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 285534, last_sequence is 4344142742, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 285390 2024/11/24-00:16:16.153140 139957248734784 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-00:16:16.153238 139957248734784 [version_set.cc:5721] Creating manifest 285534 2024/11/24-00:16:16.186970 139957248734784 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-00:16:16.208610 139957248734784 [compaction_job.cc:2437] [default] [JOB 38441] Compacting 1@2 + 11@3 files to L3, score 0.44, subcompactions 1 : 1 2024/11/24-00:16:16.208622 139957248734784 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 28801 Base level 2, inputs: [285495(25MB)], [285439(35MB) 285440(35MB) 285441(35MB) 285442(35MB) 285443(35MB) 285444(34MB) 285445(35MB) 285446(19MB) 285459(59KB) 285460(35MB) 285521(35MB)] 2024/11/24-00:16:16.215149 139957248734784 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:16:16.215182 139957248734784 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:16:16.215202 139957248734784 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:16:16.215209 139957248734784 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:16:16.215219 139957248734784 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:16:16.784585 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285535.sst) /dev/shm/ToplingTemp/Topling-224881-KMKAZE 2024/11/24-00:16:17.343736 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285535.sst): first pass time = 0.56's, 240.058'MB/sec 2024/11/24-00:16:18.937157 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285535.sst): old prealloc_size = 147639500, real_size = 32782192 2024/11/24-00:16:18.937692 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285535.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925272 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.06 avg-zval = 33.28 2024/11/24-00:16:18.938487 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285535.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:16:19.782020 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285535.sst): second pass time = 1.59's, 68.014'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.511'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3424.473'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 289.169'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26452.327'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.014'MB/sec zip pipeline throughput = 121.611'MB/sec entries = 925272 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.06 avg-zval = 33.28 usrkeys = 925272 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.06 avg-zval = 33.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925272 bytes = 3122800 } seq expand size = 6476904 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5137 value = 3.5176 dict = 1.00 all = 3.6512 } Zip/UnZip{ index = 0.0951 value = 0.2843 dict = 1.00 all = 0.2739 } ---------------------------- total value len = 8.963407 GB avg = 0.168 KB (by entry num) total key len = 1.466729 GB avg = 0.028 KB total ukey len = 1.016383 GB avg = 0.015 KB total ukey num = 0.066475707 Billion total entry num = 0.053283436 Billion write speed all = 0.188720727 MB/sec (with seq num) write speed all = 0.181007931 MB/sec (without seq num) 2024/11/24-00:16:19.828210 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285535: 925272 keys, 36760108 bytes, temperature: kWarm 2024/11/24-00:16:19.828440 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378579828377, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285535, "file_size": 36760108, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4259662629, "largest_seqno": 4343826481, "table_properties": {"data_size": 30790560, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25907616, "raw_average_key_size": 28, "raw_value_size": 108310120, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925272, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378576, "oldest_key_time": 0, "file_creation_time": 1732378576, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285535, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:19.866541 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285536.sst) /dev/shm/ToplingTemp/Topling-224881-QYl6VC 2024/11/24-00:16:20.850416 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285536.sst): first pass time = 0.98's, 136.422'MB/sec 2024/11/24-00:16:22.403953 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285536.sst): old prealloc_size = 147639500, real_size = 33528992 2024/11/24-00:16:22.404545 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285536.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925139 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.08 avg-zval = 34.08 2024/11/24-00:16:22.405302 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285536.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:16:23.257352 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285536.sst): second pass time = 1.55's, 69.759'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.951'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3185.612'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 298.748'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33946.805'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.759'MB/sec zip pipeline throughput = 99.117'MB/sec entries = 925139 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.08 avg-zval = 34.08 usrkeys = 925139 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.08 avg-zval = 34.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925139 bytes = 3122352 } seq expand size = 6475973 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4419 value = 3.4357 dict = 1.00 all = 3.5785 } Zip/UnZip{ index = 0.0958 value = 0.2911 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 9.078197 GB avg = 0.167 KB (by entry num) total key len = 1.492633 GB avg = 0.028 KB total ukey len = 1.034886 GB avg = 0.015 KB total ukey num = 0.067400846 Billion total entry num = 0.054208575 Billion write speed all = 0.191254386 MB/sec (with seq num) write speed all = 0.183408169 MB/sec (without seq num) 2024/11/24-00:16:23.304442 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285536: 925139 keys, 37506908 bytes, temperature: kWarm 2024/11/24-00:16:23.304510 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378583304475, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285536, "file_size": 37506908, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178503, "largest_seqno": 4343826731, "table_properties": {"data_size": 31525584, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903892, "raw_average_key_size": 28, "raw_value_size": 108313938, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925139, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378579, "oldest_key_time": 0, "file_creation_time": 1732378579, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285536, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:23.343332 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285537.sst) /dev/shm/ToplingTemp/Topling-224881-JZDaqF 2024/11/24-00:16:24.346786 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285537.sst): first pass time = 1.00's, 133.762'MB/sec 2024/11/24-00:16:25.943137 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285537.sst): old prealloc_size = 147639500, real_size = 32832576 2024/11/24-00:16:25.943660 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285537.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928954 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.48 avg-zval = 33.19 2024/11/24-00:16:25.944440 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285537.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:16:26.798568 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285537.sst): second pass time = 1.60's, 67.825'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.045'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3548.616'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 289.202'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32432.473'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.825'MB/sec zip pipeline throughput = 121.800'MB/sec entries = 928954 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.48 avg-zval = 33.19 usrkeys = 928954 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.48 avg-zval = 33.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928954 bytes = 3135232 } seq expand size = 6502678 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5123 value = 3.5095 dict = 1.00 all = 3.6452 } Zip/UnZip{ index = 0.0951 value = 0.2849 dict = 1.00 all = 0.2743 } ---------------------------- total value len = 9.192906 GB avg = 0.167 KB (by entry num) total key len = 1.518643 GB avg = 0.028 KB total ukey len = 1.053465 GB avg = 0.015 KB total ukey num = 0.068329800 Billion total entry num = 0.055137529 Billion write speed all = 0.193787977 MB/sec (with seq num) write speed all = 0.185807813 MB/sec (without seq num) 2024/11/24-00:16:26.843311 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285537: 928954 keys, 36820906 bytes, temperature: kWarm 2024/11/24-00:16:26.843370 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378586843339, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285537, "file_size": 36820906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178518, "largest_seqno": 4343826528, "table_properties": {"data_size": 30832816, "index_size": 1767360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26010712, "raw_average_key_size": 28, "raw_value_size": 108207072, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928954, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378583, "oldest_key_time": 0, "file_creation_time": 1732378583, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285537, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:26.880252 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285538.sst) /dev/shm/ToplingTemp/Topling-224881-cR85ZE 2024/11/24-00:16:27.747252 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285538.sst): first pass time = 0.87's, 154.815'MB/sec 2024/11/24-00:16:29.304890 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285538.sst): old prealloc_size = 147639500, real_size = 32997648 2024/11/24-00:16:29.305436 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285538.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927248 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.75 avg-zval = 33.43 2024/11/24-00:16:29.306211 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285538.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:16:30.152950 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285538.sst): second pass time = 1.56's, 69.536'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.608'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3372.742'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 287.992'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33229.293'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.536'MB/sec zip pipeline throughput = 83.093'MB/sec entries = 927248 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 33.43 usrkeys = 927248 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 33.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927248 bytes = 3129472 } seq expand size = 6490736 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5128 value = 3.4919 dict = 1.00 all = 3.6293 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 9.307652 GB avg = 0.166 KB (by entry num) total key len = 1.544606 GB avg = 0.028 KB total ukey len = 1.072010 GB avg = 0.015 KB total ukey num = 0.069257048 Billion total entry num = 0.056064777 Billion write speed all = 0.196321690 MB/sec (with seq num) write speed all = 0.188207816 MB/sec (without seq num) 2024/11/24-00:16:30.198864 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285538: 927248 keys, 36981520 bytes, temperature: kWarm 2024/11/24-00:16:30.198952 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378590198914, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285538, "file_size": 36981520, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178530, "largest_seqno": 4343826891, "table_properties": {"data_size": 31001600, "index_size": 1764032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962944, "raw_average_key_size": 28, "raw_value_size": 108254796, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927248, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378586, "oldest_key_time": 0, "file_creation_time": 1732378586, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285538, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:30.238924 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285539.sst) /dev/shm/ToplingTemp/Topling-224881-2FSbxD 2024/11/24-00:16:31.135147 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285539.sst): first pass time = 0.90's, 149.765'MB/sec 2024/11/24-00:16:32.628639 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285539.sst): old prealloc_size = 147639500, real_size = 33375680 2024/11/24-00:16:32.629231 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285539.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925133 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.08 avg-zval = 33.92 2024/11/24-00:16:32.630029 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285539.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:16:33.480476 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285539.sst): second pass time = 1.49's, 72.583'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.852'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3094.794'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 294.398'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33135.062'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.583'MB/sec zip pipeline throughput = 74.494'MB/sec entries = 925133 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 33.92 usrkeys = 925133 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 33.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925133 bytes = 3122336 } seq expand size = 6475931 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5140 value = 3.4512 dict = 1.00 all = 3.5932 } Zip/UnZip{ index = 0.0951 value = 0.2898 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 9.422442 GB avg = 0.165 KB (by entry num) total key len = 1.570510 GB avg = 0.028 KB total ukey len = 1.090513 GB avg = 0.016 KB total ukey num = 0.070182181 Billion total entry num = 0.056989910 Billion write speed all = 0.198854926 MB/sec (with seq num) write speed all = 0.190607660 MB/sec (without seq num) 2024/11/24-00:16:33.526145 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285539: 925133 keys, 37353524 bytes, temperature: kWarm 2024/11/24-00:16:33.526217 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378593526178, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285539, "file_size": 37353524, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178559, "largest_seqno": 4343827021, "table_properties": {"data_size": 31384432, "index_size": 1759808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903724, "raw_average_key_size": 28, "raw_value_size": 108314106, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925133, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378590, "oldest_key_time": 0, "file_creation_time": 1732378590, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285539, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:33.573146 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285540.sst) /dev/shm/ToplingTemp/Topling-224881-TUij7E 2024/11/24-00:16:34.458394 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285540.sst): first pass time = 0.89's, 151.623'MB/sec 2024/11/24-00:16:35.944645 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285540.sst): old prealloc_size = 147639500, real_size = 33203072 2024/11/24-00:16:35.945330 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285540.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927501 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.71 avg-zval = 33.63 2024/11/24-00:16:35.946130 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285540.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:16:36.834655 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285540.sst): second pass time = 1.49's, 72.871'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.784'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2685.226'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.15's, 246.517'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34310.565'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.871'MB/sec zip pipeline throughput = 72.871'MB/sec entries = 927501 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.71 avg-zval = 33.63 usrkeys = 927501 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.71 avg-zval = 33.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927501 bytes = 3130336 } seq expand size = 6492507 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4452 value = 3.4700 dict = 1.00 all = 3.6092 } Zip/UnZip{ index = 0.0957 value = 0.2882 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 9.537182 GB avg = 0.165 KB (by entry num) total key len = 1.596480 GB avg = 0.028 KB total ukey len = 1.109063 GB avg = 0.016 KB total ukey num = 0.071109682 Billion total entry num = 0.057917411 Billion write speed all = 0.201388059 MB/sec (with seq num) write speed all = 0.193007079 MB/sec (without seq num) 2024/11/24-00:16:36.881297 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285540: 927501 keys, 37187882 bytes, temperature: kWarm 2024/11/24-00:16:36.881377 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378596881332, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285540, "file_size": 37187882, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178575, "largest_seqno": 4343826967, "table_properties": {"data_size": 31195056, "index_size": 1775936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25970028, "raw_average_key_size": 28, "raw_value_size": 108247782, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927501, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378593, "oldest_key_time": 0, "file_creation_time": 1732378593, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285540, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:36.918204 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285541.sst) /dev/shm/ToplingTemp/Topling-224881-MkZ4eF 2024/11/24-00:16:37.943082 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285541.sst): first pass time = 1.02's, 130.964'MB/sec 2024/11/24-00:16:39.420582 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285541.sst): old prealloc_size = 147639500, real_size = 32808816 2024/11/24-00:16:39.421141 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285541.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928764 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.51 avg-zval = 33.17 2024/11/24-00:16:39.421881 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285541.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:16:40.264549 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285541.sst): second pass time = 1.48's, 73.299'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.600'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3281.500'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 289.997'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31154.832'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.299'MB/sec zip pipeline throughput = 73.299'MB/sec entries = 928764 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.51 avg-zval = 33.17 usrkeys = 928764 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.51 avg-zval = 33.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928764 bytes = 3134592 } seq expand size = 6501348 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5125 value = 3.5123 dict = 1.00 all = 3.6476 } Zip/UnZip{ index = 0.0951 value = 0.2847 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 9.651896 GB avg = 0.164 KB (by entry num) total key len = 1.622485 GB avg = 0.028 KB total ukey len = 1.127638 GB avg = 0.016 KB total ukey num = 0.072038446 Billion total entry num = 0.058846175 Billion write speed all = 0.203920766 MB/sec (with seq num) write speed all = 0.195405917 MB/sec (without seq num) 2024/11/24-00:16:40.310792 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285541: 928764 keys, 36796400 bytes, temperature: kWarm 2024/11/24-00:16:40.310863 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378600310825, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285541, "file_size": 36796400, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178589, "largest_seqno": 4343826600, "table_properties": {"data_size": 30809440, "index_size": 1766976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005392, "raw_average_key_size": 28, "raw_value_size": 108212412, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928764, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378596, "oldest_key_time": 0, "file_creation_time": 1732378596, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285541, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:40.346906 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285542.sst) /dev/shm/ToplingTemp/Topling-224881-hmOs5C 2024/11/24-00:16:41.392686 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285542.sst): first pass time = 1.05's, 86.003'MB/sec 2024/11/24-00:16:42.391507 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285542.sst): old prealloc_size = 147639500, real_size = 21771568 2024/11/24-00:16:42.391876 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285542.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 621937 prefix = 4 raw-key = 0.0100 GB zip-key = 0.0012 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0725 GB zip-val = 0.0204 GB avg-val = 116.61 avg-zval = 32.84 2024/11/24-00:16:42.392406 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285542.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0016 GB, waited 0.000 sec, Key+Value = 0.090 GB 2024/11/24-00:16:42.967896 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285542.sst): second pass time = 1.00's, 72.683'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.58's, 42.412'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3383.055'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.317'MB/sec write SST data time = 0.09's, 286.926'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.727'MB, throughput = 23644.470'MB/sec zip my value time = 1.00's, unzip length = 0.073'GB zip my value throughput = 72.683'MB/sec zip pipeline throughput = 88.307'MB/sec entries = 621937 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.61 avg-zval = 32.84 usrkeys = 621937 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.61 avg-zval = 32.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 621937 bytes = 2099056 } seq expand size = 4353559 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0124 GB value = 0.0725 GB dict = 0.73 MB all = 0.0899 GB } __ZipSize{ index = 0.0012 GB value = 0.0204 GB dict = 0.73 MB all = 0.0244 GB } UnZip/Zip{ index = 10.4402 value = 3.5508 dict = 1.00 all = 3.6792 } Zip/UnZip{ index = 0.0958 value = 0.2816 dict = 1.00 all = 0.2718 } ---------------------------- total value len = 9.728772 GB avg = 0.164 KB (by entry num) total key len = 1.639900 GB avg = 0.028 KB total ukey len = 1.140077 GB avg = 0.016 KB total ukey num = 0.072660383 Billion total entry num = 0.059468112 Billion write speed all = 0.205616146 MB/sec (with seq num) write speed all = 0.197011725 MB/sec (without seq num) 2024/11/24-00:16:43.003481 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285542: 621937 keys, 24444724 bytes, temperature: kWarm 2024/11/24-00:16:43.003557 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378603003522, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285542, "file_size": 24444724, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4276423130, "largest_seqno": 4343827072, "table_properties": {"data_size": 20424480, "index_size": 1191424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 17414236, "raw_average_key_size": 28, "raw_value_size": 72522302, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 621937, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378600, "oldest_key_time": 0, "file_creation_time": 1732378600, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285542, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:43.036389 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285543.sst) /dev/shm/ToplingTemp/Topling-224881-ED7aFE 2024/11/24-00:16:43.589727 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285543.sst): first pass time = 0.55's, 242.579'MB/sec 2024/11/24-00:16:44.922798 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285543.sst): old prealloc_size = 147639500, real_size = 32867552 2024/11/24-00:16:44.923392 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285543.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925306 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.05 avg-zval = 33.37 2024/11/24-00:16:44.924140 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285543.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:16:45.770205 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285543.sst): second pass time = 1.33's, 81.298'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.481'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3079.477'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 287.242'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31604.645'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.298'MB/sec zip pipeline throughput = 81.298'MB/sec entries = 925306 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.05 avg-zval = 33.37 usrkeys = 925306 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.05 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925306 bytes = 3122928 } seq expand size = 6477142 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5137 value = 3.5079 dict = 1.00 all = 3.6427 } Zip/UnZip{ index = 0.0951 value = 0.2851 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 9.843558 GB avg = 0.163 KB (by entry num) total key len = 1.665808 GB avg = 0.028 KB total ukey len = 1.158583 GB avg = 0.016 KB total ukey num = 0.073585689 Billion total entry num = 0.060393418 Billion write speed all = 0.208150234 MB/sec (with seq num) write speed all = 0.199412375 MB/sec (without seq num) 2024/11/24-00:16:45.817293 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285543: 925306 keys, 36845412 bytes, temperature: kWarm 2024/11/24-00:16:45.817364 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378605817328, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285543, "file_size": 36845412, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4343826831, "table_properties": {"data_size": 30875856, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25908568, "raw_average_key_size": 28, "raw_value_size": 108309264, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925306, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378603, "oldest_key_time": 0, "file_creation_time": 1732378603, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285543, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:45.854881 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285544.sst) /dev/shm/ToplingTemp/Topling-224881-rGKsbD 2024/11/24-00:16:46.714907 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285544.sst): first pass time = 0.86's, 156.070'MB/sec 2024/11/24-00:16:48.047686 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285544.sst): old prealloc_size = 147639500, real_size = 33610320 2024/11/24-00:16:48.048266 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285544.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925108 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.08 avg-zval = 34.18 2024/11/24-00:16:48.049023 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285544.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:16:48.906721 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285544.sst): second pass time = 1.33's, 81.348'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.758'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.473'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 293.838'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33617.114'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.348'MB/sec zip pipeline throughput = 81.348'MB/sec entries = 925108 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 34.18 usrkeys = 925108 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 34.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925108 bytes = 3122256 } seq expand size = 6475756 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5118 value = 3.4257 dict = 1.00 all = 3.5707 } Zip/UnZip{ index = 0.0951 value = 0.2919 dict = 1.00 all = 0.2801 } ---------------------------- total value len = 9.958349 GB avg = 0.162 KB (by entry num) total key len = 1.691711 GB avg = 0.028 KB total ukey len = 1.177085 GB avg = 0.016 KB total ukey num = 0.074510797 Billion total entry num = 0.061318526 Billion write speed all = 0.210682767 MB/sec (with seq num) write speed all = 0.201811564 MB/sec (without seq num) 2024/11/24-00:16:48.954784 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285544: 925108 keys, 37588312 bytes, temperature: kWarm 2024/11/24-00:16:48.954855 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378608954819, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285544, "file_size": 37588312, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4343826418, "table_properties": {"data_size": 31618752, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903024, "raw_average_key_size": 28, "raw_value_size": 108314780, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925108, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378605, "oldest_key_time": 0, "file_creation_time": 1732378605, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285544, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:48.994803 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285545.sst) /dev/shm/ToplingTemp/Topling-224881-RYHssD 2024/11/24-00:16:49.005899 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285545.sst): first pass time = 0.01's, 134.630'MB/sec 2024/11/24-00:16:49.038306 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285545.sst): old prealloc_size = 147639500, real_size = 500064 2024/11/24-00:16:49.038362 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285545.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 10193 prefix = 4 raw-key = 0.0002 GB zip-key = 0.0000 GB avg-key = 16.00 avg-zkey = 2.06 raw-val = 0.0012 GB zip-val = 0.0005 GB avg-val = 118.00 avg-zval = 46.73 2024/11/24-00:16:49.038413 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285545.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0000 GB, waited 0.000 sec, Key+Value = 0.001 GB 2024/11/24-00:16:49.047496 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00/285545.sst): second pass time = 0.03's, 37.817'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.01's, 58.711'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 559.652'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.366'MB/sec write SST data time = 0.00's, 247.459'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.014'MB, throughput = 457.164'MB/sec zip my value time = 0.03's, unzip length = 0.001'GB zip my value throughput = 37.817'MB/sec zip pipeline throughput = 37.817'MB/sec entries = 10193 avg-key = 28.00 avg-zkey = 2.06 avg-val = 118.00 avg-zval = 46.73 usrkeys = 10193 avg-key = 20.00 avg-zkey = 2.06 avg-val = 125.00 avg-zval = 46.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 10193 bytes = 26768 } seq expand size = 71351 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0002 GB value = 0.0012 GB dict = 0.01 MB all = 0.0015 GB } __ZipSize{ index = 0.0000 GB value = 0.0005 GB dict = 0.01 MB all = 0.0005 GB } UnZip/Zip{ index = 9.7113 value = 2.5249 dict = 1.00 all = 2.7605 } Zip/UnZip{ index = 0.1030 value = 0.3961 dict = 1.00 all = 0.3623 } ---------------------------- total value len = 9.959623 GB avg = 0.162 KB (by entry num) total key len = 1.691997 GB avg = 0.028 KB total ukey len = 1.177289 GB avg = 0.016 KB total ukey num = 0.074520990 Billion total entry num = 0.061328719 Billion write speed all = 0.210710434 MB/sec (with seq num) write speed all = 0.201837778 MB/sec (without seq num) 2024/11/24-00:16:49.054753 139957248734784 [compaction_job.cc:1948] [default] [JOB 38441] Generated table #285545: 10193 keys, 539096 bytes, temperature: kWarm 2024/11/24-00:16:49.054802 139957248734784 EVENT_LOG_v1 {"time_micros": 1732378609054775, "cf_name": "default", "job": 38441, "event": "table_file_creation", "file_number": 285545, "file_size": 539096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4334858130, "largest_seqno": 4336560027, "table_properties": {"data_size": 476368, "index_size": 20992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 285404, "raw_average_key_size": 28, "raw_value_size": 1202774, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 10193, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378608, "oldest_key_time": 0, "file_creation_time": 1732378608, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285545, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:16:49.063299 139957248734784 [compaction_job.cc:703] job-38441: subcompact[0], size: 0.358825 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38441/att-00 [285535,285536,285537,285538,285539,285540,285541,285542,285543,285544,285545] 2024/11/24-00:16:49.128674 139957248734784 (Original Log Time 2024/11/24-00:16:16.208670) EVENT_LOG_v1 {"time_micros": 1732378576208631, "job": 38441, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [285495], "files_L3": [285439, 285440, 285441, 285442, 285443, 285444, 285445, 285446, 285459, 285460, 285521], "score": 0.44115, "input_data_size": 380986839, "oldest_snapshot_seqno": 4344142722} 2024/11/24-00:16:49.128678 139957248734784 (Original Log Time 2024/11/24-00:16:49.127653) [compaction_job.cc:2022] [default] [JOB 38441] Compacted 1@2 + 11@3 files to L3 => 358824792 bytes 2024/11/24-00:16:49.128680 139957248734784 (Original Log Time 2024/11/24-00:16:49.128608) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 11.6 rd, 10.9 wr, level 3, files in(1, 11) out(11 +0 blob) MB in(25.1, 338.2 +0.0 blob) out(342.2 +0.0 blob), read-write-amplify(28.1) write-amplify(13.6) OK, records in: 9076134, records dropped: 105579 output_compression: Snappy 2024/11/24-00:16:49.128687 139957248734784 (Original Log Time 2024/11/24-00:16:49.128654) EVENT_LOG_v1 {"time_micros": 1732378609128633, "cf": "default", "job": 38441, "event": "compaction_finished", "compaction_time_micros": 32854634, "compaction_time_cpu_micros": 13681873, "output_level": 3, "num_output_files": 11, "total_output_size": 358824792, "num_input_records": 9076134, "num_output_records": 8970555, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 251635780, "file_range_sync_nanos": 0, "file_fsync_nanos": 7560, "file_prepare_write_nanos": 27020, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/24-00:16:49.255618 139957248734784 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732378576,"executesMs":33047,"compactionJobId":38441,"attempt":0,"compactionInputRawBytes":1314377104,"compactionInputZipBytes":380986839,"compactionOutputRawBytes":0,"compactionOutputZipBytes":358824792,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-de46322dd39a6924d520de7e257aa12f-98768cf2b26ef259-00"}}, response = 2024/11/24-00:16:49.255713 139957248734784 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38441/att-00: olev 3, work 33.047 s, result 7.727 ms, install 1.051 ms, input{raw 1.224 GiB zip 363.337 MiB}