2024/11/22-16:55:30.797908 140025112712768 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00: fork to child time = 0.009602 sec 2024/11/22-16:55:30.798709 140025112712768 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00 2024/11/22-16:55:30.806728 140025112712768 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00 2024/11/22-16:55:30.812506 140025112712768 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-16:55:30.812834 140025112712768 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-16:55:30.863183 140025112712768 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-05249/att-00/MANIFEST-206347 2024/11/22-16:55:30.893461 140025112712768 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-05249/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 215980, last_sequence is 3399592081, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 215801 2024/11/22-16:55:30.893469 140025112712768 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-16:55:30.893567 140025112712768 [version_set.cc:5721] Creating manifest 215980 2024/11/22-16:55:30.925548 140025112712768 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-16:55:30.952787 140025112712768 [compaction_job.cc:2437] [default] [JOB 5249] Compacting 1@2 + 15@3 files to L3, score 0.98, subcompactions 1 : 1 2024/11/22-16:55:30.952804 140025112712768 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 4067 Base level 2, inputs: [215943(27MB)], [215886(35MB) 215887(36MB) 215888(36MB) 215889(35MB) 215890(35MB) 215891(35MB) 215892(36MB) 215893(38MB) 215894(38MB) 215895(37MB) 215896(35MB) 215897(35MB) 215898(36MB) 215899(37MB) 215973(37MB)] 2024/11/22-16:55:30.958687 140025112712768 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:55:30.958716 140025112712768 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:55:30.958734 140025112712768 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:55:30.958738 140025112712768 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:55:30.958751 140025112712768 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:55:31.471284 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215981.sst) /dev/shm/ToplingTemp/Topling-208627-qlw10A 2024/11/22-16:55:32.019632 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215981.sst): first pass time = 0.55's, 244.788'MB/sec 2024/11/22-16:55:33.464979 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215981.sst): old prealloc_size = 147639500, real_size = 33712304 2024/11/22-16:55:33.465592 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215981.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921450 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.66 avg-zval = 34.42 2024/11/22-16:55:33.466399 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215981.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:55:34.408021 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215981.sst): second pass time = 1.44's, 75.054'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.957'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2985.858'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.15's, 258.232'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32598.887'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.054'MB/sec zip pipeline throughput = 75.054'MB/sec entries = 921450 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.66 avg-zval = 34.42 usrkeys = 921450 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.66 avg-zval = 34.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921450 bytes = 3109904 } seq expand size = 6450150 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4433 value = 3.4183 dict = 1.00 all = 3.5619 } Zip/UnZip{ index = 0.0958 value = 0.2925 dict = 1.00 all = 0.2807 } ---------------------------- total value len = 0.114867 GB avg = 0.125 KB (by entry num) total key len = 0.025801 GB avg = 0.028 KB total ukey len = 0.018429 GB avg = 0.020 KB total ukey num = 0.000921450 Billion total entry num = 0.000921450 Billion write speed all = 0.004852071 MB/sec (with seq num) write speed all = 0.004597801 MB/sec (without seq num) 2024/11/22-16:55:34.443308 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215981: 921450 keys, 37681462 bytes, temperature: kWarm 2024/11/22-16:55:34.443394 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265734443342, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215981, "file_size": 37681462, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702550, "largest_seqno": 3399262754, "table_properties": {"data_size": 31717088, "index_size": 1764672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25800600, "raw_average_key_size": 28, "raw_value_size": 108417140, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921450, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265731, "oldest_key_time": 0, "file_creation_time": 1732265731, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215981, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:55:34.481330 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215982.sst) /dev/shm/ToplingTemp/Topling-208627-efPraA 2024/11/22-16:55:35.306467 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215982.sst): first pass time = 0.83's, 162.668'MB/sec 2024/11/22-16:55:36.848548 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215982.sst): old prealloc_size = 147639500, real_size = 33982208 2024/11/22-16:55:36.849109 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215982.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921533 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.65 avg-zval = 34.72 2024/11/22-16:55:36.849861 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215982.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:55:37.776607 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215982.sst): second pass time = 1.54's, 70.361'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.894'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3238.266'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 298.686'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31612.999'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.361'MB/sec zip pipeline throughput = 111.733'MB/sec entries = 921533 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.65 avg-zval = 34.72 usrkeys = 921533 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.65 avg-zval = 34.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921533 bytes = 3110192 } seq expand size = 6450731 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5144 value = 3.3881 dict = 1.00 all = 3.5366 } Zip/UnZip{ index = 0.0951 value = 0.2952 dict = 1.00 all = 0.2828 } ---------------------------- total value len = 0.229733 GB avg = 0.125 KB (by entry num) total key len = 0.051604 GB avg = 0.028 KB total ukey len = 0.036860 GB avg = 0.020 KB total ukey num = 0.001842983 Billion total entry num = 0.001842983 Billion write speed all = 0.009703034 MB/sec (with seq num) write speed all = 0.009194532 MB/sec (without seq num) 2024/11/22-16:55:37.812504 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215982: 921533 keys, 37951518 bytes, temperature: kWarm 2024/11/22-16:55:37.812561 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265737812529, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215982, "file_size": 37951518, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702561, "largest_seqno": 3399262961, "table_properties": {"data_size": 31998768, "index_size": 1752896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25802924, "raw_average_key_size": 28, "raw_value_size": 108414830, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921533, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265734, "oldest_key_time": 0, "file_creation_time": 1732265734, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215982, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:55:37.849483 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215983.sst) /dev/shm/ToplingTemp/Topling-208627-6kFSVz 2024/11/22-16:55:38.719547 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215983.sst): first pass time = 0.87's, 154.269'MB/sec 2024/11/22-16:55:40.246329 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215983.sst): old prealloc_size = 147639500, real_size = 33878352 2024/11/22-16:55:40.246893 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215983.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923598 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.32 avg-zval = 34.53 2024/11/22-16:55:40.247632 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215983.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:55:41.151728 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215983.sst): second pass time = 1.53's, 71.044'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.809'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3256.251'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.13's, 299.828'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33735.282'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 71.044'MB/sec zip pipeline throughput = 107.030'MB/sec entries = 923598 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.32 avg-zval = 34.53 usrkeys = 923598 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.32 avg-zval = 34.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923598 bytes = 3117152 } seq expand size = 6465186 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5123 value = 3.3978 dict = 1.00 all = 3.5458 } Zip/UnZip{ index = 0.0951 value = 0.2943 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 0.344555 GB avg = 0.125 KB (by entry num) total key len = 0.077464 GB avg = 0.028 KB total ukey len = 0.055332 GB avg = 0.020 KB total ukey num = 0.002766581 Billion total entry num = 0.002766581 Billion write speed all = 0.014553369 MB/sec (with seq num) write speed all = 0.013790123 MB/sec (without seq num) 2024/11/22-16:55:41.187339 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215983: 923598 keys, 37852912 bytes, temperature: kWarm 2024/11/22-16:55:41.187400 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265741187366, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215983, "file_size": 37852912, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702575, "largest_seqno": 3399262634, "table_properties": {"data_size": 31890112, "index_size": 1757184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25860744, "raw_average_key_size": 28, "raw_value_size": 108357092, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923598, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265737, "oldest_key_time": 0, "file_creation_time": 1732265737, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215983, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:55:41.221973 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215984.sst) /dev/shm/ToplingTemp/Topling-208627-IKXuwC 2024/11/22-16:55:42.099262 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215984.sst): first pass time = 0.88's, 152.999'MB/sec 2024/11/22-16:55:43.733387 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215984.sst): old prealloc_size = 147639500, real_size = 33807376 2024/11/22-16:55:43.733944 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215984.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921350 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.68 avg-zval = 34.54 2024/11/22-16:55:43.734684 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215984.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:55:44.656422 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215984.sst): second pass time = 1.63's, 66.374'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.927'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3275.394'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.13's, 301.185'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 29074.393'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.374'MB/sec zip pipeline throughput = 132.742'MB/sec entries = 921350 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.68 avg-zval = 34.54 usrkeys = 921350 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.68 avg-zval = 34.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921350 bytes = 3109568 } seq expand size = 6449450 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5131 value = 3.4069 dict = 1.00 all = 3.5530 } Zip/UnZip{ index = 0.0951 value = 0.2935 dict = 1.00 all = 0.2815 } ---------------------------- total value len = 0.459425 GB avg = 0.125 KB (by entry num) total key len = 0.103262 GB avg = 0.028 KB total ukey len = 0.073759 GB avg = 0.020 KB total ukey num = 0.003687931 Billion total entry num = 0.003687931 Billion write speed all = 0.019401941 MB/sec (with seq num) write speed all = 0.018384636 MB/sec (without seq num) 2024/11/22-16:55:44.691288 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215984: 921350 keys, 37776180 bytes, temperature: kWarm 2024/11/22-16:55:44.691347 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265744691316, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215984, "file_size": 37776180, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702604, "largest_seqno": 3399262409, "table_properties": {"data_size": 31824064, "index_size": 1752768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25797800, "raw_average_key_size": 28, "raw_value_size": 108419928, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921350, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265741, "oldest_key_time": 0, "file_creation_time": 1732265741, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215984, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:55:44.728502 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215985.sst) /dev/shm/ToplingTemp/Topling-208627-4LEIYB 2024/11/22-16:55:45.543055 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215985.sst): first pass time = 0.81's, 164.782'MB/sec 2024/11/22-16:55:47.077489 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215985.sst): old prealloc_size = 147639500, real_size = 33763248 2024/11/22-16:55:47.078085 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215985.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921431 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.66 avg-zval = 34.49 2024/11/22-16:55:47.078845 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215985.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:55:48.005695 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215985.sst): second pass time = 1.53's, 70.689'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.651'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3060.977'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.287'MB/sec write SST data time = 0.13's, 299.156'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32784.426'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.689'MB/sec zip pipeline throughput = 112.084'MB/sec entries = 921431 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.66 avg-zval = 34.49 usrkeys = 921431 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.66 avg-zval = 34.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921431 bytes = 3109840 } seq expand size = 6450017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5132 value = 3.4115 dict = 1.00 all = 3.5571 } Zip/UnZip{ index = 0.0951 value = 0.2931 dict = 1.00 all = 0.2811 } ---------------------------- total value len = 0.574292 GB avg = 0.125 KB (by entry num) total key len = 0.129062 GB avg = 0.028 KB total ukey len = 0.092187 GB avg = 0.020 KB total ukey num = 0.004609362 Billion total entry num = 0.004609362 Billion write speed all = 0.024249492 MB/sec (with seq num) write speed all = 0.022978160 MB/sec (without seq num) 2024/11/22-16:55:48.038219 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215985: 921431 keys, 37732314 bytes, temperature: kWarm 2024/11/22-16:55:48.038283 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265748038250, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215985, "file_size": 37732314, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702617, "largest_seqno": 3399261945, "table_properties": {"data_size": 31779808, "index_size": 1752896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25800068, "raw_average_key_size": 28, "raw_value_size": 108417678, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265744, "oldest_key_time": 0, "file_creation_time": 1732265744, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215985, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:55:48.076210 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215986.sst) /dev/shm/ToplingTemp/Topling-208627-QaQDaz 2024/11/22-16:55:48.972321 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215986.sst): first pass time = 0.90's, 149.784'MB/sec 2024/11/22-16:55:50.521645 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215986.sst): old prealloc_size = 147639500, real_size = 33785584 2024/11/22-16:55:50.522205 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215986.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923576 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.32 avg-zval = 34.43 2024/11/22-16:55:50.522971 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215986.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:55:51.418288 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215986.sst): second pass time = 1.55's, 69.970'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.114'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3340.607'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.13's, 299.299'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31752.339'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.970'MB/sec zip pipeline throughput = 126.308'MB/sec entries = 923576 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.32 avg-zval = 34.43 usrkeys = 923576 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.32 avg-zval = 34.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923576 bytes = 3117088 } seq expand size = 6465032 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5105 value = 3.4078 dict = 1.00 all = 3.5545 } Zip/UnZip{ index = 0.0951 value = 0.2934 dict = 1.00 all = 0.2813 } ---------------------------- total value len = 0.689115 GB avg = 0.125 KB (by entry num) total key len = 0.154922 GB avg = 0.028 KB total ukey len = 0.110659 GB avg = 0.020 KB total ukey num = 0.005532938 Billion total entry num = 0.005532938 Billion write speed all = 0.029096378 MB/sec (with seq num) write speed all = 0.027570488 MB/sec (without seq num) 2024/11/22-16:55:51.456182 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215986: 923576 keys, 37760244 bytes, temperature: kWarm 2024/11/22-16:55:51.456246 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265751456211, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215986, "file_size": 37760244, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702634, "largest_seqno": 3399262733, "table_properties": {"data_size": 31797088, "index_size": 1757440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25860128, "raw_average_key_size": 28, "raw_value_size": 108357620, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923576, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265748, "oldest_key_time": 0, "file_creation_time": 1732265748, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215986, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:55:51.492975 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215987.sst) /dev/shm/ToplingTemp/Topling-208627-9tBYaC 2024/11/22-16:55:52.382621 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215987.sst): first pass time = 0.89's, 150.872'MB/sec 2024/11/22-16:55:53.933643 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215987.sst): old prealloc_size = 147639500, real_size = 33860448 2024/11/22-16:55:53.934239 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215987.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921406 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.67 avg-zval = 34.58 2024/11/22-16:55:53.935012 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215987.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:55:54.875827 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215987.sst): second pass time = 1.55's, 69.930'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.151'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3064.833'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.14's, 263.210'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 26507.704'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.930'MB/sec zip pipeline throughput = 120.486'MB/sec entries = 921406 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.67 avg-zval = 34.58 usrkeys = 921406 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.67 avg-zval = 34.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921406 bytes = 3109760 } seq expand size = 6449842 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4436 value = 3.4024 dict = 1.00 all = 3.5480 } Zip/UnZip{ index = 0.0958 value = 0.2939 dict = 1.00 all = 0.2819 } ---------------------------- total value len = 0.803983 GB avg = 0.125 KB (by entry num) total key len = 0.180722 GB avg = 0.028 KB total ukey len = 0.129087 GB avg = 0.020 KB total ukey num = 0.006454344 Billion total entry num = 0.006454344 Billion write speed all = 0.033941549 MB/sec (with seq num) write speed all = 0.032161763 MB/sec (without seq num) 2024/11/22-16:55:54.911004 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215987: 921406 keys, 37829474 bytes, temperature: kWarm 2024/11/22-16:55:54.911067 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265754911035, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215987, "file_size": 37829474, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702661, "largest_seqno": 3399261857, "table_properties": {"data_size": 31865360, "index_size": 1764544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25799368, "raw_average_key_size": 28, "raw_value_size": 108418412, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921406, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265751, "oldest_key_time": 0, "file_creation_time": 1732265751, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215987, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:55:54.956929 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215988.sst) /dev/shm/ToplingTemp/Topling-208627-doa86z 2024/11/22-16:55:55.834464 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215988.sst): first pass time = 0.88's, 152.956'MB/sec 2024/11/22-16:55:57.512393 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215988.sst): old prealloc_size = 147639500, real_size = 36533920 2024/11/22-16:55:57.513009 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215988.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923099 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0345 GB avg-val = 117.40 avg-zval = 37.42 2024/11/22-16:55:57.513772 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215988.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:55:58.395025 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215988.sst): second pass time = 1.68's, 64.615'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 46.029'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2946.137'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.14's, 283.035'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31305.270'MB/sec zip my value time = 1.68's, unzip length = 0.108'GB zip my value throughput = 64.615'MB/sec zip pipeline throughput = 121.851'MB/sec entries = 923099 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.40 avg-zval = 37.42 usrkeys = 923099 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.40 avg-zval = 37.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923099 bytes = 3230864 } seq expand size = 6461693 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0406 GB } UnZip/Zip{ index = 10.5112 value = 3.1370 dict = 1.00 all = 3.3037 } Zip/UnZip{ index = 0.0951 value = 0.3188 dict = 1.00 all = 0.3027 } ---------------------------- total value len = 0.918816 GB avg = 0.125 KB (by entry num) total key len = 0.206568 GB avg = 0.028 KB total ukey len = 0.147549 GB avg = 0.020 KB total ukey num = 0.007377443 Billion total entry num = 0.007377443 Billion write speed all = 0.038785890 MB/sec (with seq num) write speed all = 0.036751807 MB/sec (without seq num) 2024/11/22-16:55:58.438373 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215988: 923099 keys, 40626386 bytes, temperature: kWarm 2024/11/22-16:55:58.438436 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265758438403, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215988, "file_size": 40626386, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3399261698, "table_properties": {"data_size": 34546576, "index_size": 1756416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25846772, "raw_average_key_size": 28, "raw_value_size": 108370998, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923099, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265754, "oldest_key_time": 0, "file_creation_time": 1732265754, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215988, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:55:58.476900 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215989.sst) /dev/shm/ToplingTemp/Topling-208627-9My9UB 2024/11/22-16:55:59.296816 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215989.sst): first pass time = 0.82's, 163.705'MB/sec 2024/11/22-16:56:00.802091 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215989.sst): old prealloc_size = 147639500, real_size = 36488272 2024/11/22-16:56:00.802642 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215989.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923189 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0345 GB avg-val = 117.39 avg-zval = 37.36 2024/11/22-16:56:00.803385 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215989.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:56:01.648151 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215989.sst): second pass time = 1.50's, 72.060'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 47.789'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3344.961'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 309.130'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33620.538'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.060'MB/sec zip pipeline throughput = 108.320'MB/sec entries = 923189 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.39 avg-zval = 37.36 usrkeys = 923189 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.16 avg-zval = 37.36 TagRS{ kinds = RS_Key0_TagN bytes = 144360 } TagArray{ size = 894151 bytes = 2906000 } seq expand size = 6259057 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.4467 value = 3.1420 dict = 1.00 all = 3.3225 } Zip/UnZip{ index = 0.0957 value = 0.3183 dict = 1.00 all = 0.3010 } ---------------------------- total value len = 1.033443 GB avg = 0.125 KB (by entry num) total key len = 0.232418 GB avg = 0.028 KB total ukey len = 0.166013 GB avg = 0.020 KB total ukey num = 0.008300632 Billion total entry num = 0.008300632 Billion write speed all = 0.043622477 MB/sec (with seq num) write speed all = 0.041334112 MB/sec (without seq num) 2024/11/22-16:56:01.685918 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215989: 923189 keys, 40396732 bytes, temperature: kWarm 2024/11/22-16:56:01.685975 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265761685944, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215989, "file_size": 40396732, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273699977, "table_properties": {"data_size": 34489856, "index_size": 1767424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25849292, "raw_average_key_size": 28, "raw_value_size": 108368550, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923189, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265758, "oldest_key_time": 0, "file_creation_time": 1732265758, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215989, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:56:01.726214 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215990.sst) /dev/shm/ToplingTemp/Topling-208627-sRmfBA 2024/11/22-16:56:02.619624 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215990.sst): first pass time = 0.89's, 150.238'MB/sec 2024/11/22-16:56:03.973995 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215990.sst): old prealloc_size = 147639500, real_size = 36480976 2024/11/22-16:56:03.974546 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215990.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923480 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1084 GB zip-val = 0.0345 GB avg-val = 117.34 avg-zval = 37.36 2024/11/22-16:56:03.975131 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215990.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:56:04.760279 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215990.sst): second pass time = 1.35's, 80.072'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 50.402'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3306.052'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.352'MB/sec write SST data time = 0.13's, 304.442'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30375.193'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.072'MB/sec zip pipeline throughput = 80.072'MB/sec entries = 923480 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.34 avg-zval = 37.36 usrkeys = 923480 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.34 avg-zval = 37.36 TagRS{ kinds = RS_Key0_TagN bytes = 144400 } TagArray{ size = 659163 bytes = 2142288 } seq expand size = 4614141 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0345 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5822 value = 3.1405 dict = 1.00 all = 3.3868 } Zip/UnZip{ index = 0.0945 value = 0.3184 dict = 1.00 all = 0.2953 } ---------------------------- total value len = 1.146418 GB avg = 0.124 KB (by entry num) total key len = 0.258275 GB avg = 0.028 KB total ukey len = 0.184482 GB avg = 0.020 KB total ukey num = 0.009224112 Billion total entry num = 0.009224112 Billion write speed all = 0.048401533 MB/sec (with seq num) write speed all = 0.045858850 MB/sec (without seq num) 2024/11/22-16:56:04.796420 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215990: 923480 keys, 39629226 bytes, temperature: kWarm 2024/11/22-16:56:04.796494 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265764796453, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215990, "file_size": 39629226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 34504576, "index_size": 1745344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25857440, "raw_average_key_size": 28, "raw_value_size": 108360368, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923480, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265761, "oldest_key_time": 0, "file_creation_time": 1732265761, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215990, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:56:04.836273 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215991.sst) /dev/shm/ToplingTemp/Topling-208627-miTbkz 2024/11/22-16:56:05.593544 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215991.sst): first pass time = 0.76's, 177.249'MB/sec 2024/11/22-16:56:07.158372 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215991.sst): old prealloc_size = 147639500, real_size = 36480192 2024/11/22-16:56:07.158929 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215991.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927141 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.21 2024/11/22-16:56:07.158971 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215991.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:56:07.801132 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215991.sst): second pass time = 1.56's, 69.210'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.092'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3288.531'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.448'MB/sec write SST data time = 0.12's, 299.863'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31104.874'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.210'MB/sec zip pipeline throughput = 69.210'MB/sec entries = 927141 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.21 usrkeys = 927141 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.21 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5826 value = 3.1383 dict = 1.00 all = 3.5946 } Zip/UnZip{ index = 0.0945 value = 0.3186 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 1.254676 GB avg = 0.124 KB (by entry num) total key len = 0.284235 GB avg = 0.028 KB total ukey len = 0.203025 GB avg = 0.020 KB total ukey num = 0.010151253 Billion total entry num = 0.010151253 Billion write speed all = 0.053020723 MB/sec (with seq num) write speed all = 0.050222761 MB/sec (without seq num) 2024/11/22-16:56:07.836444 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215991: 927141 keys, 37338412 bytes, temperature: kWarm 2024/11/22-16:56:07.836505 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265767836474, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215991, "file_size": 37338412, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34496048, "index_size": 1752192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959948, "raw_average_key_size": 28, "raw_value_size": 108257862, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927141, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265764, "oldest_key_time": 0, "file_creation_time": 1732265764, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215991, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:56:07.879214 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215992.sst) /dev/shm/ToplingTemp/Topling-208627-9VQjQB 2024/11/22-16:56:08.686937 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215992.sst): first pass time = 0.81's, 166.177'MB/sec 2024/11/22-16:56:10.270157 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215992.sst): old prealloc_size = 147639500, real_size = 36404800 2024/11/22-16:56:10.270696 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215992.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926924 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.80 avg-zval = 37.12 2024/11/22-16:56:10.270739 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215992.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:56:10.913812 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215992.sst): second pass time = 1.58's, 68.428'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 57.895'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3420.973'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.446'MB/sec write SST data time = 0.12's, 301.025'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29934.543'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.428'MB/sec zip pipeline throughput = 68.428'MB/sec entries = 926924 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 37.12 usrkeys = 926924 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 37.12 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5130 value = 3.1463 dict = 1.00 all = 3.6019 } Zip/UnZip{ index = 0.0951 value = 0.3178 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 1.362940 GB avg = 0.123 KB (by entry num) total key len = 0.310189 GB avg = 0.028 KB total ukey len = 0.221564 GB avg = 0.020 KB total ukey num = 0.011078177 Billion total entry num = 0.011078177 Billion write speed all = 0.057638800 MB/sec (with seq num) write speed all = 0.054585680 MB/sec (without seq num) 2024/11/22-16:56:10.949480 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215992: 926924 keys, 37262782 bytes, temperature: kWarm 2024/11/22-16:56:10.949542 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265770949509, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215992, "file_size": 37262782, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34409520, "index_size": 1763392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953872, "raw_average_key_size": 28, "raw_value_size": 108263860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926924, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265767, "oldest_key_time": 0, "file_creation_time": 1732265767, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215992, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:56:10.984886 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215993.sst) /dev/shm/ToplingTemp/Topling-208627-WCRR8y 2024/11/22-16:56:11.822540 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215993.sst): first pass time = 0.84's, 160.238'MB/sec 2024/11/22-16:56:13.166613 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215993.sst): old prealloc_size = 147639500, real_size = 34819984 2024/11/22-16:56:13.167189 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215993.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924734 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0328 GB avg-val = 117.14 avg-zval = 35.51 2024/11/22-16:56:13.167881 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215993.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:56:14.053700 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215993.sst): second pass time = 1.34's, 80.664'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.439'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3167.934'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.13's, 294.810'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29742.035'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.664'MB/sec zip pipeline throughput = 80.664'MB/sec entries = 924734 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.14 avg-zval = 35.51 usrkeys = 924734 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.22 avg-zval = 35.51 TagRS{ kinds = RS_Key0_TagN bytes = 144600 } TagArray{ size = 802910 bytes = 2709840 } seq expand size = 5620370 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0328 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5807 value = 3.2985 dict = 1.00 all = 3.4831 } Zip/UnZip{ index = 0.0945 value = 0.3032 dict = 1.00 all = 0.2871 } ---------------------------- total value len = 1.476885 GB avg = 0.123 KB (by entry num) total key len = 0.336082 GB avg = 0.028 KB total ukey len = 0.240058 GB avg = 0.020 KB total ukey num = 0.012002911 Billion total entry num = 0.012002911 Billion write speed all = 0.062449430 MB/sec (with seq num) write speed all = 0.059141813 MB/sec (without seq num) 2024/11/22-16:56:14.088366 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215993: 924734 keys, 38533756 bytes, temperature: kWarm 2024/11/22-16:56:14.088431 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265774088398, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215993, "file_size": 38533756, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3399262985, "table_properties": {"data_size": 32840640, "index_size": 1747968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25892552, "raw_average_key_size": 28, "raw_value_size": 108325240, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924734, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265770, "oldest_key_time": 0, "file_creation_time": 1732265770, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215993, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:56:14.124692 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215994.sst) /dev/shm/ToplingTemp/Topling-208627-9JbUaz 2024/11/22-16:56:15.049147 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215994.sst): first pass time = 0.92's, 145.192'MB/sec 2024/11/22-16:56:16.406077 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215994.sst): old prealloc_size = 147639500, real_size = 34547168 2024/11/22-16:56:16.406661 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215994.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923729 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0325 GB avg-val = 117.30 avg-zval = 35.23 2024/11/22-16:56:16.407431 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215994.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:56:17.315869 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215994.sst): second pass time = 1.36's, 79.890'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.343'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3164.502'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.295'MB/sec write SST data time = 0.13's, 303.972'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32181.559'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.890'MB/sec zip pipeline throughput = 79.890'MB/sec entries = 923729 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.30 avg-zval = 35.23 usrkeys = 923729 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.30 avg-zval = 35.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923729 bytes = 3117600 } seq expand size = 6466103 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4422 value = 3.3292 dict = 1.00 all = 3.4841 } Zip/UnZip{ index = 0.0958 value = 0.3004 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 1.591705 GB avg = 0.123 KB (by entry num) total key len = 0.361946 GB avg = 0.028 KB total ukey len = 0.258533 GB avg = 0.020 KB total ukey num = 0.012926640 Billion total entry num = 0.012926640 Billion write speed all = 0.067287867 MB/sec (with seq num) write speed all = 0.063726100 MB/sec (without seq num) 2024/11/22-16:56:17.350033 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215994: 923729 keys, 38522702 bytes, temperature: kWarm 2024/11/22-16:56:17.350093 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265777350062, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215994, "file_size": 38522702, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3399263038, "table_properties": {"data_size": 32546832, "index_size": 1769216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25864412, "raw_average_key_size": 28, "raw_value_size": 108353446, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923729, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265774, "oldest_key_time": 0, "file_creation_time": 1732265774, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215994, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:56:17.389147 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215995.sst) /dev/shm/ToplingTemp/Topling-208627-4p4qGz 2024/11/22-16:56:18.345355 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215995.sst): first pass time = 0.96's, 140.371'MB/sec 2024/11/22-16:56:19.689636 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215995.sst): old prealloc_size = 147639500, real_size = 35954288 2024/11/22-16:56:19.690203 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215995.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930548 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0340 GB avg-val = 116.24 avg-zval = 36.49 2024/11/22-16:56:19.690920 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215995.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:56:20.530042 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215995.sst): second pass time = 1.34's, 80.500'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.256'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3247.489'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 314.714'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31170.087'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.500'MB/sec zip pipeline throughput = 80.500'MB/sec entries = 930548 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.24 avg-zval = 36.49 usrkeys = 930548 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.24 avg-zval = 36.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930548 bytes = 2907984 } seq expand size = 6513836 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5140 value = 3.1858 dict = 1.00 all = 3.3796 } Zip/UnZip{ index = 0.0951 value = 0.3139 dict = 1.00 all = 0.2959 } ---------------------------- total value len = 1.706381 GB avg = 0.123 KB (by entry num) total key len = 0.388001 GB avg = 0.028 KB total ukey len = 0.277144 GB avg = 0.020 KB total ukey num = 0.013857188 Billion total entry num = 0.013857188 Billion write speed all = 0.072126976 MB/sec (with seq num) write speed all = 0.068309231 MB/sec (without seq num) 2024/11/22-16:56:20.567461 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215995: 930548 keys, 39713770 bytes, temperature: kWarm 2024/11/22-16:56:20.567520 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265780567489, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215995, "file_size": 39713770, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774735, "largest_seqno": 3399263100, "table_properties": {"data_size": 33951328, "index_size": 1770112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26055344, "raw_average_key_size": 28, "raw_value_size": 108162404, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930548, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265777, "oldest_key_time": 0, "file_creation_time": 1732265777, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215995, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:56:20.614140 140025112712768 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215996.sst) /dev/shm/ToplingTemp/Topling-208627-y163aA 2024/11/22-16:56:20.689382 140025112712768 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215996.sst): first pass time = 0.08's, 234.343'MB/sec 2024/11/22-16:56:20.855279 140025112712768 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215996.sst): old prealloc_size = 147639500, real_size = 5022416 2024/11/22-16:56:20.855407 140025112712768 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215996.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 122981 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0142 GB zip-val = 0.0048 GB avg-val = 115.29 avg-zval = 38.66 2024/11/22-16:56:20.855532 140025112712768 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215996.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/22-16:56:20.963541 140025112712768 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00/215996.sst): second pass time = 0.17's, 85.771'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.11's, 50.862'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2316.105'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.340'MB/sec write SST data time = 0.02's, 312.384'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.148'MB, throughput = 4920.867'MB/sec zip my value time = 0.17's, unzip length = 0.014'GB zip my value throughput = 85.771'MB/sec zip pipeline throughput = 85.771'MB/sec entries = 122981 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.29 avg-zval = 38.66 usrkeys = 122981 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.83 avg-zval = 38.66 TagRS{ kinds = RS_Key0_TagN bytes = 19320 } TagArray{ size = 114961 bytes = 344896 } seq expand size = 804727 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0025 GB value = 0.0142 GB dict = 0.15 MB all = 0.0176 GB } __ZipSize{ index = 0.0002 GB value = 0.0048 GB dict = 0.15 MB all = 0.0055 GB } UnZip/Zip{ index = 10.3897 value = 2.9819 dict = 1.00 all = 3.2001 } Zip/UnZip{ index = 0.0962 value = 0.3354 dict = 1.00 all = 0.3125 } ---------------------------- total value len = 1.721364 GB avg = 0.123 KB (by entry num) total key len = 0.391445 GB avg = 0.028 KB total ukey len = 0.279603 GB avg = 0.020 KB total ukey num = 0.013980169 Billion total entry num = 0.013980169 Billion write speed all = 0.072760455 MB/sec (with seq num) write speed all = 0.068908886 MB/sec (without seq num) 2024/11/22-16:56:20.973889 140025112712768 [compaction_job.cc:1948] [default] [JOB 5249] Generated table #215996: 122981 keys, 5506533 bytes, temperature: kWarm 2024/11/22-16:56:20.973943 140025112712768 EVENT_LOG_v1 {"time_micros": 1732265780973914, "cf_name": "default", "job": 5249, "event": "table_file_creation", "file_number": 215996, "file_size": 5506533, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3399263120, "table_properties": {"data_size": 4754752, "index_size": 236736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3443468, "raw_average_key_size": 28, "raw_value_size": 14177966, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 122981, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265780, "oldest_key_time": 0, "file_creation_time": 1732265780, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215996, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:56:20.983809 140025112712768 [compaction_job.cc:703] job-05249: subcompact[0], size: 0.582114 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-05249/att-00 [215981,215982,215983,215984,215985,215986,215987,215988,215989,215990,215991,215992,215993,215994,215995,215996] 2024/11/22-16:56:21.081914 140025112712768 (Original Log Time 2024/11/22-16:55:30.952848) EVENT_LOG_v1 {"time_micros": 1732265730952816, "job": 5249, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [215943], "files_L3": [215886, 215887, 215888, 215889, 215890, 215891, 215892, 215893, 215894, 215895, 215896, 215897, 215898, 215899, 215973], "score": 0.981862, "input_data_size": 605572301, "oldest_snapshot_seqno": 3399592049} 2024/11/22-16:56:21.081918 140025112712768 (Original Log Time 2024/11/22-16:56:21.081130) [compaction_job.cc:2022] [default] [JOB 5249] Compacted 1@2 + 15@3 files to L3 => 582114403 bytes 2024/11/22-16:56:21.081919 140025112712768 (Original Log Time 2024/11/22-16:56:21.081850) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 12.1 rd, 11.6 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.1, 550.4 +0.0 blob) out(555.1 +0.0 blob), read-write-amplify(41.8) write-amplify(20.5) OK, records in: 14092071, records dropped: 111902 output_compression: Snappy 2024/11/22-16:56:21.081925 140025112712768 (Original Log Time 2024/11/22-16:56:21.081894) EVENT_LOG_v1 {"time_micros": 1732265781081874, "cf": "default", "job": 5249, "event": "compaction_finished", "compaction_time_micros": 50030968, "compaction_time_cpu_micros": 21317142, "output_level": 3, "num_output_files": 16, "total_output_size": 582114403, "num_input_records": 14092071, "num_output_records": 13980169, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 361141229, "file_range_sync_nanos": 0, "file_fsync_nanos": 15560, "file_prepare_write_nanos": 33827, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-16:56:21.225806 140025112712768 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732265730,"executesMs":50278,"compactionJobId":5249,"attempt":0,"compactionInputRawBytes":2046778210,"compactionInputZipBytes":605572301,"compactionOutputRawBytes":0,"compactionOutputZipBytes":582114403,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-e7d2b1930ee3b67508c8ae9e9634de3d-96478fba3c1fa9c1-00"}}, response = 2024/11/22-16:56:21.225884 140025112712768 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-05249/att-00: olev 3, work 50.278 s, result 4.902 ms, install 0.814 ms, input{raw 1.906 GiB zip 577.519 MiB}