2024/11/22-16:12:57.725967 140024902895168 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00: fork to child time = 0.009219 sec 2024/11/22-16:12:57.726761 140024902895168 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00 2024/11/22-16:12:57.733644 140024902895168 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00 2024/11/22-16:12:57.734773 140024902895168 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-16:12:57.735143 140024902895168 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-16:12:57.783892 140024902895168 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04146/att-00/MANIFEST-206347 2024/11/22-16:12:57.797043 140024902895168 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04146/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 213739, last_sequence is 3368125350, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 213557 2024/11/22-16:12:57.797050 140024902895168 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-16:12:57.797130 140024902895168 [version_set.cc:5721] Creating manifest 213739 2024/11/22-16:12:57.826662 140024902895168 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-16:12:57.846051 140024902895168 [compaction_job.cc:2437] [default] [JOB 4146] Compacting 1@2 + 11@3 files to L3, score 0.87, subcompactions 1 : 1 2024/11/22-16:12:57.846068 140024902895168 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 3249 Base level 2, inputs: [213687(27MB)], [213712(36MB) 213713(169KB) 213625(36MB) 213626(36MB) 213627(34MB) 213628(36MB) 213629(36MB) 213630(36MB) 213631(36MB) 213640(36MB) 213730(36MB)] 2024/11/22-16:12:57.852031 140024902895168 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:12:57.852057 140024902895168 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:12:57.852074 140024902895168 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:12:57.852079 140024902895168 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:12:57.852089 140024902895168 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:12:58.309475 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213740.sst) /dev/shm/ToplingTemp/Topling-193003-QdxLfd 2024/11/22-16:12:58.871029 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213740.sst): first pass time = 0.56's, 239.031'MB/sec 2024/11/22-16:13:00.475903 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213740.sst): old prealloc_size = 147639500, real_size = 33853888 2024/11/22-16:13:00.476439 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213740.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922558 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.48 avg-zval = 34.53 2024/11/22-16:13:00.477237 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213740.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:13:01.573041 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213740.sst): second pass time = 1.60's, 67.571'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 34.477'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3427.846'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.237'MB/sec write SST data time = 0.13's, 301.583'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 38850.339'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.571'MB/sec zip pipeline throughput = 67.571'MB/sec entries = 922558 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.48 avg-zval = 34.53 usrkeys = 922558 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.48 avg-zval = 34.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922558 bytes = 3113648 } seq expand size = 6457906 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4434 value = 3.4023 dict = 1.00 all = 3.5483 } Zip/UnZip{ index = 0.0958 value = 0.2939 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 0.114844 GB avg = 0.124 KB (by entry num) total key len = 0.025832 GB avg = 0.028 KB total ukey len = 0.018451 GB avg = 0.020 KB total ukey num = 0.000922558 Billion total entry num = 0.000922558 Billion write speed all = 0.005320871 MB/sec (with seq num) write speed all = 0.005041715 MB/sec (without seq num) 2024/11/22-16:13:01.606822 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213740: 922558 keys, 37826064 bytes, temperature: kWarm 2024/11/22-16:13:01.606910 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263181606858, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213740, "file_size": 37826064, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701431, "largest_seqno": 3367674840, "table_properties": {"data_size": 31856304, "index_size": 1766784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25831624, "raw_average_key_size": 28, "raw_value_size": 108386192, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922558, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263178, "oldest_key_time": 0, "file_creation_time": 1732263178, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213740, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:01.647212 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213741.sst) /dev/shm/ToplingTemp/Topling-193003-dkAddc 2024/11/22-16:13:02.730637 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213741.sst): first pass time = 1.08's, 123.888'MB/sec 2024/11/22-16:13:04.161860 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213741.sst): old prealloc_size = 147639500, real_size = 34010640 2024/11/22-16:13:04.162370 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213741.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921246 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.69 avg-zval = 34.77 2024/11/22-16:13:04.163134 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213741.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:13:05.077607 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213741.sst): second pass time = 1.43's, 75.793'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.474'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3559.694'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.291'MB/sec write SST data time = 0.12's, 304.475'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33441.304'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.793'MB/sec zip pipeline throughput = 75.793'MB/sec entries = 921246 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.69 avg-zval = 34.77 usrkeys = 921246 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.69 avg-zval = 34.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921246 bytes = 3109216 } seq expand size = 6448722 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5142 value = 3.3853 dict = 1.00 all = 3.5340 } Zip/UnZip{ index = 0.0951 value = 0.2954 dict = 1.00 all = 0.2830 } ---------------------------- total value len = 0.229716 GB avg = 0.125 KB (by entry num) total key len = 0.051627 GB avg = 0.028 KB total ukey len = 0.036876 GB avg = 0.020 KB total ukey num = 0.001843804 Billion total entry num = 0.001843804 Billion write speed all = 0.010639981 MB/sec (with seq num) write speed all = 0.010082140 MB/sec (without seq num) 2024/11/22-16:13:05.111934 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213741: 921246 keys, 37979008 bytes, temperature: kWarm 2024/11/22-16:13:05.111997 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263185111964, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213741, "file_size": 37979008, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701445, "largest_seqno": 3367675134, "table_properties": {"data_size": 32027776, "index_size": 1752384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25794888, "raw_average_key_size": 28, "raw_value_size": 108422852, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921246, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263181, "oldest_key_time": 0, "file_creation_time": 1732263181, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213741, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:05.148647 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213742.sst) /dev/shm/ToplingTemp/Topling-193003-gDHpLc 2024/11/22-16:13:06.028172 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213742.sst): first pass time = 0.88's, 152.609'MB/sec 2024/11/22-16:13:07.494039 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213742.sst): old prealloc_size = 147639500, real_size = 35015008 2024/11/22-16:13:07.494559 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213742.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925128 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0330 GB avg-val = 117.08 avg-zval = 35.70 2024/11/22-16:13:07.495169 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213742.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:13:08.346726 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213742.sst): second pass time = 1.46's, 73.948'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.878'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3528.426'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 295.592'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34477.680'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 73.948'MB/sec zip pipeline throughput = 73.948'MB/sec entries = 925128 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 35.70 usrkeys = 925128 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.12 avg-zval = 35.70 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 665919 bytes = 2247488 } seq expand size = 4661433 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5128 value = 3.2799 dict = 1.00 all = 3.5075 } Zip/UnZip{ index = 0.0951 value = 0.3049 dict = 1.00 all = 0.2851 } ---------------------------- total value len = 0.342691 GB avg = 0.124 KB (by entry num) total key len = 0.077530 GB avg = 0.028 KB total ukey len = 0.055379 GB avg = 0.020 KB total ukey num = 0.002768932 Billion total entry num = 0.002768932 Billion write speed all = 0.015890242 MB/sec (with seq num) write speed all = 0.015052607 MB/sec (without seq num) 2024/11/22-16:13:08.380566 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213742: 925128 keys, 38265856 bytes, temperature: kWarm 2024/11/22-16:13:08.380630 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263188380597, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213742, "file_size": 38265856, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3367674995, "table_properties": {"data_size": 33023568, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903584, "raw_average_key_size": 28, "raw_value_size": 108314232, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925128, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263185, "oldest_key_time": 0, "file_creation_time": 1732263185, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213742, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:08.421038 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213743.sst) /dev/shm/ToplingTemp/Topling-193003-7hbR5c 2024/11/22-16:13:09.318836 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213743.sst): first pass time = 0.90's, 149.503'MB/sec 2024/11/22-16:13:10.855815 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213743.sst): old prealloc_size = 147639500, real_size = 34929664 2024/11/22-16:13:10.856424 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213743.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081414 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.11 avg-zval = 30.15 2024/11/22-16:13:10.856464 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213743.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:13:11.479144 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213743.sst): second pass time = 1.54's, 67.663'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.62's, 57.283'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3501.346'MB/sec (index lex order gen) rebuild zvType time = 0.48's, 0.558'MB/sec write SST data time = 0.14's, 258.368'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.036'MB, throughput = 34220.082'MB/sec zip my value time = 1.54's, unzip length = 0.104'GB zip my value throughput = 67.663'MB/sec zip pipeline throughput = 67.663'MB/sec entries = 1081414 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.11 avg-zval = 30.15 usrkeys = 1081414 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.11 avg-zval = 30.15 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.04 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5170 value = 3.1880 dict = 1.00 all = 3.7590 } Zip/UnZip{ index = 0.0951 value = 0.3137 dict = 1.00 all = 0.2660 } ---------------------------- total value len = 0.446629 GB avg = 0.116 KB (by entry num) total key len = 0.107810 GB avg = 0.028 KB total ukey len = 0.077007 GB avg = 0.020 KB total ukey num = 0.003850346 Billion total entry num = 0.003850346 Billion write speed all = 0.020963065 MB/sec (with seq num) write speed all = 0.019798428 MB/sec (without seq num) 2024/11/22-16:13:11.509987 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213743: 1081414 keys, 35705915 bytes, temperature: kWarm 2024/11/22-16:13:11.510048 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263191510016, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213743, "file_size": 35705915, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32602608, "index_size": 2056512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30279592, "raw_average_key_size": 28, "raw_value_size": 103938141, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081414, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263188, "oldest_key_time": 0, "file_creation_time": 1732263188, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213743, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:11.544797 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213744.sst) /dev/shm/ToplingTemp/Topling-193003-UM0Hub 2024/11/22-16:13:12.435065 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213744.sst): first pass time = 0.89's, 150.768'MB/sec 2024/11/22-16:13:13.878686 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213744.sst): old prealloc_size = 147639500, real_size = 34320624 2024/11/22-16:13:13.879213 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213744.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921501 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0323 GB avg-val = 117.65 avg-zval = 35.08 2024/11/22-16:13:13.879959 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213744.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:13:14.779565 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213744.sst): second pass time = 1.44's, 75.138'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.489'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3468.350'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.299'MB/sec write SST data time = 0.13's, 295.330'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 37570.864'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.138'MB/sec zip pipeline throughput = 75.138'MB/sec entries = 921501 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.65 avg-zval = 35.08 usrkeys = 921501 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.29 avg-zval = 35.08 TagRS{ kinds = RS_Key0_TagN bytes = 144080 } TagArray{ size = 873721 bytes = 2948816 } seq expand size = 6116047 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4439 value = 3.3539 dict = 1.00 all = 3.5065 } Zip/UnZip{ index = 0.0957 value = 0.2982 dict = 1.00 all = 0.2852 } ---------------------------- total value len = 0.561161 GB avg = 0.118 KB (by entry num) total key len = 0.133612 GB avg = 0.028 KB total ukey len = 0.095437 GB avg = 0.020 KB total ukey num = 0.004771847 Billion total entry num = 0.004771847 Billion write speed all = 0.026265739 MB/sec (with seq num) write speed all = 0.024822550 MB/sec (without seq num) 2024/11/22-16:13:14.814538 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213744: 921501 keys, 38276350 bytes, temperature: kWarm 2024/11/22-16:13:14.814596 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263194814563, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213744, "file_size": 38276350, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3367674555, "table_properties": {"data_size": 32325408, "index_size": 1764672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25802028, "raw_average_key_size": 28, "raw_value_size": 108415770, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921501, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263191, "oldest_key_time": 0, "file_creation_time": 1732263191, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213744, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:14.848110 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213745.sst) /dev/shm/ToplingTemp/Topling-193003-XoOBX9 2024/11/22-16:13:15.634469 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213745.sst): first pass time = 0.79's, 170.690'MB/sec 2024/11/22-16:13:17.193537 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213745.sst): old prealloc_size = 147639500, real_size = 34547136 2024/11/22-16:13:17.194077 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213745.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922720 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0326 GB avg-val = 117.46 avg-zval = 35.29 2024/11/22-16:13:17.194842 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213745.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:13:18.097244 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213745.sst): second pass time = 1.56's, 69.547'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.625'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3362.202'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.298'MB/sec write SST data time = 0.13's, 300.147'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33945.599'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.547'MB/sec zip pipeline throughput = 115.269'MB/sec entries = 922720 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 35.29 usrkeys = 922720 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 35.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922720 bytes = 3114192 } seq expand size = 6459040 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5130 value = 3.3286 dict = 1.00 all = 3.4844 } Zip/UnZip{ index = 0.0951 value = 0.3004 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 0.676002 GB avg = 0.119 KB (by entry num) total key len = 0.159448 GB avg = 0.028 KB total ukey len = 0.113891 GB avg = 0.020 KB total ukey num = 0.005694567 Billion total entry num = 0.005694567 Billion write speed all = 0.031580034 MB/sec (with seq num) write speed all = 0.029857996 MB/sec (without seq num) 2024/11/22-16:13:18.130152 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213745: 922720 keys, 38519734 bytes, temperature: kWarm 2024/11/22-16:13:18.130208 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263198130178, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213745, "file_size": 38519734, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701084, "largest_seqno": 3367673945, "table_properties": {"data_size": 32560880, "index_size": 1755392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25836160, "raw_average_key_size": 28, "raw_value_size": 108381636, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922720, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263194, "oldest_key_time": 0, "file_creation_time": 1732263194, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213745, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:18.165583 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213746.sst) /dev/shm/ToplingTemp/Topling-193003-vd9elc 2024/11/22-16:13:19.081856 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213746.sst): first pass time = 0.92's, 146.487'MB/sec 2024/11/22-16:13:20.514017 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213746.sst): old prealloc_size = 147639500, real_size = 33910752 2024/11/22-16:13:20.514516 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213746.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921112 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.71 avg-zval = 34.66 2024/11/22-16:13:20.515274 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213746.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:13:21.434963 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213746.sst): second pass time = 1.43's, 75.758'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.131'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3652.973'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 302.053'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35340.291'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.758'MB/sec zip pipeline throughput = 75.758'MB/sec entries = 921112 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.71 avg-zval = 34.66 usrkeys = 921112 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.71 avg-zval = 34.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921112 bytes = 3108768 } seq expand size = 6447784 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5115 value = 3.3960 dict = 1.00 all = 3.5434 } Zip/UnZip{ index = 0.0951 value = 0.2945 dict = 1.00 all = 0.2822 } ---------------------------- total value len = 0.790876 GB avg = 0.120 KB (by entry num) total key len = 0.185239 GB avg = 0.028 KB total ukey len = 0.132314 GB avg = 0.020 KB total ukey num = 0.006615679 Billion total entry num = 0.006615679 Billion write speed all = 0.036892543 MB/sec (with seq num) write speed all = 0.034892212 MB/sec (without seq num) 2024/11/22-16:13:21.472196 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213746: 921112 keys, 37878628 bytes, temperature: kWarm 2024/11/22-16:13:21.472256 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263201472225, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213746, "file_size": 37878628, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701122, "largest_seqno": 3367675162, "table_properties": {"data_size": 31927696, "index_size": 1752576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25791136, "raw_average_key_size": 28, "raw_value_size": 108426644, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921112, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263198, "oldest_key_time": 0, "file_creation_time": 1732263198, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213746, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:21.509637 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213747.sst) /dev/shm/ToplingTemp/Topling-193003-7gb06b 2024/11/22-16:13:22.414336 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213747.sst): first pass time = 0.90's, 148.361'MB/sec 2024/11/22-16:13:23.858628 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213747.sst): old prealloc_size = 147639500, real_size = 34421872 2024/11/22-16:13:23.859149 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213747.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922705 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0324 GB avg-val = 117.46 avg-zval = 35.14 2024/11/22-16:13:23.859913 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213747.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:13:24.761122 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213747.sst): second pass time = 1.44's, 75.077'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.543'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3541.206'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 303.190'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 25624.689'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.077'MB/sec zip pipeline throughput = 75.077'MB/sec entries = 922705 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.46 avg-zval = 35.14 usrkeys = 922705 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.46 avg-zval = 35.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922705 bytes = 3114144 } seq expand size = 6458935 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4428 value = 3.3427 dict = 1.00 all = 3.4958 } Zip/UnZip{ index = 0.0958 value = 0.2992 dict = 1.00 all = 0.2861 } ---------------------------- total value len = 0.905717 GB avg = 0.120 KB (by entry num) total key len = 0.211075 GB avg = 0.028 KB total ukey len = 0.150768 GB avg = 0.020 KB total ukey num = 0.007538384 Billion total entry num = 0.007538384 Billion write speed all = 0.042204153 MB/sec (with seq num) write speed all = 0.039925117 MB/sec (without seq num) 2024/11/22-16:13:24.795332 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213747: 922705 keys, 38394250 bytes, temperature: kWarm 2024/11/22-16:13:24.795393 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263204795362, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213747, "file_size": 38394250, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701135, "largest_seqno": 3367675052, "table_properties": {"data_size": 32423840, "index_size": 1767168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835740, "raw_average_key_size": 28, "raw_value_size": 108382066, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922705, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263201, "oldest_key_time": 0, "file_creation_time": 1732263201, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213747, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:24.830223 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213748.sst) /dev/shm/ToplingTemp/Topling-193003-x7mAxa 2024/11/22-16:13:25.746736 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213748.sst): first pass time = 0.92's, 146.450'MB/sec 2024/11/22-16:13:27.211569 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213748.sst): old prealloc_size = 147639500, real_size = 34538640 2024/11/22-16:13:27.212117 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213748.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922757 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0326 GB avg-val = 117.45 avg-zval = 35.28 2024/11/22-16:13:27.212879 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213748.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:13:28.114632 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213748.sst): second pass time = 1.46's, 74.035'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.646'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3325.633'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.13's, 307.134'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 41395.315'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.035'MB/sec zip pipeline throughput = 74.035'MB/sec entries = 922757 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.45 avg-zval = 35.28 usrkeys = 922757 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.45 avg-zval = 35.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922757 bytes = 3114320 } seq expand size = 6459299 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5149 value = 3.3294 dict = 1.00 all = 3.4851 } Zip/UnZip{ index = 0.0951 value = 0.3004 dict = 1.00 all = 0.2869 } ---------------------------- total value len = 1.020557 GB avg = 0.121 KB (by entry num) total key len = 0.236912 GB avg = 0.028 KB total ukey len = 0.169223 GB avg = 0.020 KB total ukey num = 0.008461141 Billion total entry num = 0.008461141 Billion write speed all = 0.047514394 MB/sec (with seq num) write speed all = 0.044956711 MB/sec (without seq num) 2024/11/22-16:13:28.147692 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213748: 922757 keys, 38511386 bytes, temperature: kWarm 2024/11/22-16:13:28.147757 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263208147722, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213748, "file_size": 38511386, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701165, "largest_seqno": 3367674331, "table_properties": {"data_size": 32552640, "index_size": 1755136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25837196, "raw_average_key_size": 28, "raw_value_size": 108380662, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922757, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263204, "oldest_key_time": 0, "file_creation_time": 1732263204, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213748, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:28.182974 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213749.sst) /dev/shm/ToplingTemp/Topling-193003-eQPtw9 2024/11/22-16:13:28.988267 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213749.sst): first pass time = 0.81's, 166.676'MB/sec 2024/11/22-16:13:30.418732 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213749.sst): old prealloc_size = 147639500, real_size = 34007664 2024/11/22-16:13:30.419304 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213749.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921046 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.72 avg-zval = 34.75 2024/11/22-16:13:30.421179 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213749.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:13:31.363131 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213749.sst): second pass time = 1.43's, 75.837'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.212'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3216.110'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.281'MB/sec write SST data time = 0.13's, 303.287'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33436.160'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.837'MB/sec zip pipeline throughput = 75.837'MB/sec entries = 921046 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.72 avg-zval = 34.75 usrkeys = 921046 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.72 avg-zval = 34.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921046 bytes = 3108544 } seq expand size = 6447322 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.3763 value = 3.3882 dict = 1.00 all = 3.5343 } Zip/UnZip{ index = 0.0964 value = 0.2951 dict = 1.00 all = 0.2829 } ---------------------------- total value len = 1.135433 GB avg = 0.121 KB (by entry num) total key len = 0.262701 GB avg = 0.028 KB total ukey len = 0.187644 GB avg = 0.020 KB total ukey num = 0.009382187 Billion total entry num = 0.009382187 Billion write speed all = 0.052823044 MB/sec (with seq num) write speed all = 0.049987290 MB/sec (without seq num) 2024/11/22-16:13:31.395617 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213749: 921046 keys, 37975692 bytes, temperature: kWarm 2024/11/22-16:13:31.395678 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263211395645, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213749, "file_size": 37975692, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701203, "largest_seqno": 3367674353, "table_properties": {"data_size": 32001952, "index_size": 1775296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25789288, "raw_average_key_size": 28, "raw_value_size": 108428508, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921046, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263208, "oldest_key_time": 0, "file_creation_time": 1732263208, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213749, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:31.429855 140024902895168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213750.sst) /dev/shm/ToplingTemp/Topling-193003-XbKSk9 2024/11/22-16:13:31.508982 140024902895168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213750.sst): first pass time = 0.08's, 225.221'MB/sec 2024/11/22-16:13:31.656554 140024902895168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213750.sst): old prealloc_size = 147639500, real_size = 5101456 2024/11/22-16:13:31.656670 140024902895168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213750.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 124114 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0143 GB zip-val = 0.0048 GB avg-val = 115.53 avg-zval = 38.94 2024/11/22-16:13:31.656794 140024902895168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213750.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/22-16:13:31.758490 140024902895168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00/213750.sst): second pass time = 0.15's, 97.510'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.10's, 54.187'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2489.917'MB/sec (index lex order gen) rebuild zvType time = 0.08's, 0.368'MB/sec write SST data time = 0.02's, 315.193'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.150'MB, throughput = 5861.149'MB/sec zip my value time = 0.15's, unzip length = 0.014'GB zip my value throughput = 97.510'MB/sec zip pipeline throughput = 97.510'MB/sec entries = 124114 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.53 avg-zval = 38.94 usrkeys = 124114 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.80 avg-zval = 38.94 TagRS{ kinds = RS_Key0_TagN bytes = 19480 } TagArray{ size = 93391 bytes = 280192 } seq expand size = 653737 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0025 GB value = 0.0143 GB dict = 0.15 MB all = 0.0178 GB } __ZipSize{ index = 0.0002 GB value = 0.0048 GB dict = 0.15 MB all = 0.0055 GB } UnZip/Zip{ index = 10.4487 value = 2.9670 dict = 1.00 all = 3.2256 } Zip/UnZip{ index = 0.0957 value = 0.3370 dict = 1.00 all = 0.3100 } ---------------------------- total value len = 1.150426 GB avg = 0.121 KB (by entry num) total key len = 0.266176 GB avg = 0.028 KB total ukey len = 0.190126 GB avg = 0.020 KB total ukey num = 0.009506301 Billion total entry num = 0.009506301 Billion write speed all = 0.053519977 MB/sec (with seq num) write speed all = 0.050646753 MB/sec (without seq num) 2024/11/22-16:13:31.768283 140024902895168 [compaction_job.cc:1948] [default] [JOB 4146] Generated table #213750: 124114 keys, 5522705 bytes, temperature: kWarm 2024/11/22-16:13:31.768334 140024902895168 EVENT_LOG_v1 {"time_micros": 1732263211768307, "cf_name": "default", "job": 4146, "event": "table_file_creation", "file_number": 213750, "file_size": 5522705, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3367673106, "table_properties": {"data_size": 4832704, "index_size": 237568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3475192, "raw_average_key_size": 28, "raw_value_size": 14338852, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 124114, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732263211, "oldest_key_time": 0, "file_creation_time": 1732263211, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 213750, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:13:31.779883 140024902895168 [compaction_job.cc:703] job-04146: subcompact[0], size: 0.384856 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04146/att-00 [213740,213741,213742,213743,213744,213745,213746,213747,213748,213749,213750] 2024/11/22-16:13:31.851181 140024902895168 (Original Log Time 2024/11/22-16:12:57.846115) EVENT_LOG_v1 {"time_micros": 1732263177846081, "job": 4146, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [213687], "files_L3": [213712, 213713, 213625, 213626, 213627, 213628, 213629, 213630, 213631, 213640, 213730], "score": 0.874671, "input_data_size": 407569335, "oldest_snapshot_seqno": 3368125350} 2024/11/22-16:13:31.851184 140024902895168 (Original Log Time 2024/11/22-16:13:31.850554) [compaction_job.cc:2022] [default] [JOB 4146] Compacted 1@2 + 11@3 files to L3 => 384855588 bytes 2024/11/22-16:13:31.851185 140024902895168 (Original Log Time 2024/11/22-16:13:31.851117) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 12.0 rd, 11.3 wr, level 3, files in(1, 11) out(11 +0 blob) MB in(27.1, 361.6 +0.0 blob) out(367.0 +0.0 blob), read-write-amplify(27.9) write-amplify(13.5) OK, records in: 9619711, records dropped: 113410 output_compression: Snappy 2024/11/22-16:13:31.851191 140024902895168 (Original Log Time 2024/11/22-16:13:31.851162) EVENT_LOG_v1 {"time_micros": 1732263211851142, "cf": "default", "job": 4146, "event": "compaction_finished", "compaction_time_micros": 33933775, "compaction_time_cpu_micros": 14848110, "output_level": 3, "num_output_files": 11, "total_output_size": 384855588, "num_input_records": 9619711, "num_output_records": 9506301, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 241656185, "file_range_sync_nanos": 0, "file_fsync_nanos": 10846, "file_prepare_write_nanos": 22719, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/22-16:13:31.978603 140024902895168 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732263177,"executesMs":34118,"compactionJobId":4146,"attempt":0,"compactionInputRawBytes":1376096203,"compactionInputZipBytes":407569335,"compactionOutputRawBytes":0,"compactionOutputZipBytes":384855588,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-df63845ba5af7aa3b5666cbf7c7033f2-0eb33399f9739919-00"}}, response = 2024/11/22-16:13:31.978672 140024902895168 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04146/att-00: olev 3, work 34.118 s, result 6.287 ms, install 0.655 ms, input{raw 1.282 GiB zip 388.688 MiB}