2024/11/23-19:05:45.067340 139957685155392 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00: fork to child time = 0.026103 sec 2024/11/23-19:05:45.068314 139957685155392 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00 2024/11/23-19:05:45.076713 139957685155392 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00 2024/11/23-19:05:45.077945 139957685155392 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-19:05:45.078327 139957685155392 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-19:05:45.134536 139957685155392 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33807/att-00/MANIFEST-206347 2024/11/23-19:05:45.150391 139957685155392 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33807/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 276368, last_sequence is 4204283388, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 276214 2024/11/23-19:05:45.150400 139957685155392 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-19:05:45.150594 139957685155392 [version_set.cc:5721] Creating manifest 276368 2024/11/23-19:05:45.186353 139957685155392 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-19:05:45.209110 139957685155392 [compaction_job.cc:2437] [default] [JOB 33807] Compacting 1@2 + 18@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-19:05:45.209137 139957685155392 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 25316 Base level 2, inputs: [276348(27MB)], [276365(37MB) 276366(1911KB) 276290(37MB) 276291(38MB) 276292(37MB) 276294(38MB) 276295(35MB) 276296(34MB) 276297(33MB) 276298(35MB) 276299(35MB) 276300(36MB) 276301(34MB) 276302(35MB) 276303(35MB) 276304(37MB) 276317(37MB) 276318(36MB)] 2024/11/23-19:05:45.217732 139957685155392 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:05:45.217775 139957685155392 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:05:45.217800 139957685155392 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:05:45.217809 139957685155392 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:05:45.217825 139957685155392 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:05:45.820826 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276369.sst) /dev/shm/ToplingTemp/Topling-160298-agkzpc 2024/11/23-19:05:46.408723 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276369.sst): first pass time = 0.59's, 228.318'MB/sec 2024/11/23-19:05:47.855536 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276369.sst): old prealloc_size = 147639500, real_size = 35438560 2024/11/23-19:05:47.856119 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276369.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928102 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0334 GB avg-val = 116.62 avg-zval = 36.03 2024/11/23-19:05:47.856864 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276369.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:05:49.142354 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276369.sst): second pass time = 1.45's, 74.849'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.29's, 30.638'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3131.845'MB/sec (index lex order gen) rebuild zvType time = 1.04's, 0.224'MB/sec write SST data time = 0.25's, 157.907'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34740.669'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.849'MB/sec zip pipeline throughput = 74.849'MB/sec entries = 928102 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.62 avg-zval = 36.03 usrkeys = 928102 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.62 avg-zval = 36.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928102 bytes = 3132352 } seq expand size = 6496714 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.5134 value = 3.2365 dict = 1.00 all = 3.4044 } Zip/UnZip{ index = 0.0951 value = 0.3090 dict = 1.00 all = 0.2937 } ---------------------------- total value len = 8.963347 GB avg = 0.168 KB (by entry num) total key len = 1.466808 GB avg = 0.028 KB total ukey len = 1.016440 GB avg = 0.015 KB total ukey num = 0.066478537 Billion total entry num = 0.053286266 Billion write speed all = 0.284689685 MB/sec (with seq num) write speed all = 0.273054154 MB/sec (without seq num) 2024/11/23-19:05:49.195261 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276369: 928102 keys, 39424460 bytes, temperature: kWarm 2024/11/23-19:05:49.195385 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359949195309, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276369, "file_size": 39424460, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4136224572, "largest_seqno": 4204202295, "table_properties": {"data_size": 33440784, "index_size": 1765568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986856, "raw_average_key_size": 28, "raw_value_size": 108230968, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928102, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359945, "oldest_key_time": 0, "file_creation_time": 1732359945, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276369, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:05:49.278325 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276370.sst) /dev/shm/ToplingTemp/Topling-160298-XUKkqd 2024/11/23-19:05:50.702865 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276370.sst): first pass time = 1.42's, 94.222'MB/sec 2024/11/23-19:05:52.943224 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276370.sst): old prealloc_size = 147639500, real_size = 35723984 2024/11/23-19:05:52.943796 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276370.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928145 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 116.61 avg-zval = 36.34 2024/11/23-19:05:52.944561 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276370.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:05:53.888528 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276370.sst): second pass time = 2.24's, 48.327'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 41.887'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3215.926'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.23's, 172.163'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35163.686'MB/sec zip my value time = 2.24's, unzip length = 0.108'GB zip my value throughput = 48.327'MB/sec zip pipeline throughput = 98.122'MB/sec entries = 928145 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.61 avg-zval = 36.34 usrkeys = 928145 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 36.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928145 bytes = 3016480 } seq expand size = 6497015 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5112 value = 3.2091 dict = 1.00 all = 3.3899 } Zip/UnZip{ index = 0.0951 value = 0.3116 dict = 1.00 all = 0.2950 } ---------------------------- total value len = 9.078074 GB avg = 0.167 KB (by entry num) total key len = 1.492796 GB avg = 0.028 KB total ukey len = 1.035003 GB avg = 0.015 KB total ukey num = 0.067406682 Billion total entry num = 0.054214411 Billion write speed all = 0.288493106 MB/sec (with seq num) write speed all = 0.276656439 MB/sec (without seq num) 2024/11/23-19:05:53.946906 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276370: 928145 keys, 39593746 bytes, temperature: kWarm 2024/11/23-19:05:53.947010 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359953946950, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276370, "file_size": 39593746, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4148263343, "largest_seqno": 4204202338, "table_properties": {"data_size": 33725760, "index_size": 1766016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25988060, "raw_average_key_size": 28, "raw_value_size": 108229798, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928145, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359949, "oldest_key_time": 0, "file_creation_time": 1732359949, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276370, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:05:54.016972 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276371.sst) /dev/shm/ToplingTemp/Topling-160298-uQihvb 2024/11/23-19:05:55.110682 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276371.sst): first pass time = 1.09's, 122.727'MB/sec 2024/11/23-19:05:56.726569 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276371.sst): old prealloc_size = 147639500, real_size = 34846256 2024/11/23-19:05:56.727119 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276371.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929274 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0328 GB avg-val = 116.43 avg-zval = 35.35 2024/11/23-19:05:56.727912 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276371.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:05:57.771075 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276371.sst): second pass time = 1.62's, 66.993'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 37.519'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.582'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.23's, 166.881'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34159.169'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.993'MB/sec zip pipeline throughput = 87.969'MB/sec entries = 929274 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.43 avg-zval = 35.35 usrkeys = 929274 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.43 avg-zval = 35.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929274 bytes = 3484800 } seq expand size = 6504918 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5125 value = 3.2941 dict = 1.00 all = 3.4250 } Zip/UnZip{ index = 0.0951 value = 0.3036 dict = 1.00 all = 0.2920 } ---------------------------- total value len = 9.192777 GB avg = 0.167 KB (by entry num) total key len = 1.518816 GB avg = 0.028 KB total ukey len = 1.053588 GB avg = 0.015 KB total ukey num = 0.068335956 Billion total entry num = 0.055143685 Billion write speed all = 0.292302643 MB/sec (with seq num) write speed all = 0.280264362 MB/sec (without seq num) 2024/11/23-19:05:57.824629 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276371: 929274 keys, 39187294 bytes, temperature: kWarm 2024/11/23-19:05:57.824719 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359957824672, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276371, "file_size": 39187294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4204202364, "table_properties": {"data_size": 32845792, "index_size": 1767936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26019672, "raw_average_key_size": 28, "raw_value_size": 108198104, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929274, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359954, "oldest_key_time": 0, "file_creation_time": 1732359954, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276371, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:05:57.896506 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276372.sst) /dev/shm/ToplingTemp/Topling-160298-9TrOD9 2024/11/23-19:05:58.896444 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276372.sst): first pass time = 1.00's, 134.233'MB/sec 2024/11/23-19:06:00.492460 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276372.sst): old prealloc_size = 147639500, real_size = 36667680 2024/11/23-19:06:00.492971 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276372.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932630 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0346 GB avg-val = 115.91 avg-zval = 37.14 2024/11/23-19:06:00.493620 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276372.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:01.499393 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276372.sst): second pass time = 1.60's, 67.769'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 39.740'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3580.693'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.19's, 207.671'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35621.666'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.769'MB/sec zip pipeline throughput = 67.769'MB/sec entries = 932630 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.91 avg-zval = 37.14 usrkeys = 932630 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.61 avg-zval = 37.14 TagRS{ kinds = RS_Key0_TagN bytes = 145840 } TagArray{ size = 625838 bytes = 2346912 } seq expand size = 4380866 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.3773 value = 3.1211 dict = 1.00 all = 3.3542 } Zip/UnZip{ index = 0.0964 value = 0.3204 dict = 1.00 all = 0.2981 } ---------------------------- total value len = 9.305262 GB avg = 0.166 KB (by entry num) total key len = 1.544929 GB avg = 0.028 KB total ukey len = 1.072241 GB avg = 0.015 KB total ukey num = 0.069268586 Billion total entry num = 0.056076315 Billion write speed all = 0.296054663 MB/sec (with seq num) write speed all = 0.283814028 MB/sec (without seq num) 2024/11/23-19:06:01.549967 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276372: 932630 keys, 40014788 bytes, temperature: kWarm 2024/11/23-19:06:01.550053 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359961550010, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276372, "file_size": 40014788, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34636880, "index_size": 1797440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26113640, "raw_average_key_size": 28, "raw_value_size": 108104160, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932630, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359957, "oldest_key_time": 0, "file_creation_time": 1732359957, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276372, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:01.586130 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276373.sst) /dev/shm/ToplingTemp/Topling-160298-61TXEb 2024/11/23-19:06:02.547964 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276373.sst): first pass time = 0.96's, 139.550'MB/sec 2024/11/23-19:06:04.149165 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276373.sst): old prealloc_size = 147639500, real_size = 35747504 2024/11/23-19:06:04.149737 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276373.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928708 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 116.52 avg-zval = 36.33 2024/11/23-19:06:04.150554 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276373.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:05.236213 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276373.sst): second pass time = 1.60's, 67.616'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 36.767'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3236.005'MB/sec (index lex order gen) rebuild zvType time = 0.96's, 0.243'MB/sec write SST data time = 0.13's, 308.376'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 26949.192'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.616'MB/sec zip pipeline throughput = 67.616'MB/sec entries = 928708 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.52 avg-zval = 36.33 usrkeys = 928708 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.52 avg-zval = 36.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928708 bytes = 3366576 } seq expand size = 6500956 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4434 value = 3.2076 dict = 1.00 all = 3.3582 } Zip/UnZip{ index = 0.0958 value = 0.3118 dict = 1.00 all = 0.2978 } ---------------------------- total value len = 9.419977 GB avg = 0.165 KB (by entry num) total key len = 1.570933 GB avg = 0.028 KB total ukey len = 1.090815 GB avg = 0.016 KB total ukey num = 0.070197294 Billion total entry num = 0.057005023 Billion write speed all = 0.299863691 MB/sec (with seq num) write speed all = 0.287421601 MB/sec (without seq num) 2024/11/23-19:06:05.286360 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276373: 928708 keys, 39967092 bytes, temperature: kWarm 2024/11/23-19:06:05.286422 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359965286388, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276373, "file_size": 39967092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33736608, "index_size": 1778560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003824, "raw_average_key_size": 28, "raw_value_size": 108213916, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928708, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359961, "oldest_key_time": 0, "file_creation_time": 1732359961, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276373, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:05.326194 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276374.sst) /dev/shm/ToplingTemp/Topling-160298-lHPfgb 2024/11/23-19:06:06.226530 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276374.sst): first pass time = 0.90's, 149.083'MB/sec 2024/11/23-19:06:07.755686 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276374.sst): old prealloc_size = 147639500, real_size = 32592320 2024/11/23-19:06:07.756249 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276374.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927695 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.68 avg-zval = 32.98 2024/11/23-19:06:07.757083 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276374.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:08.811265 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276374.sst): second pass time = 1.53's, 70.831'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 34.982'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3277.948'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.257'MB/sec write SST data time = 0.15's, 239.530'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 40007.999'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.831'MB/sec zip pipeline throughput = 70.831'MB/sec entries = 927695 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 32.98 usrkeys = 927695 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927695 bytes = 3478864 } seq expand size = 6493865 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5145 value = 3.5378 dict = 1.00 all = 3.6348 } Zip/UnZip{ index = 0.0951 value = 0.2827 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 9.534713 GB avg = 0.165 KB (by entry num) total key len = 1.596909 GB avg = 0.028 KB total ukey len = 1.109369 GB avg = 0.016 KB total ukey num = 0.071124989 Billion total entry num = 0.057932718 Billion write speed all = 0.303673091 MB/sec (with seq num) write speed all = 0.291029753 MB/sec (without seq num) 2024/11/23-19:06:08.858275 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276374: 927695 keys, 36925512 bytes, temperature: kWarm 2024/11/23-19:06:08.858343 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359968858308, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276374, "file_size": 36925512, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30595632, "index_size": 1764608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975460, "raw_average_key_size": 28, "raw_value_size": 108242342, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927695, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359965, "oldest_key_time": 0, "file_creation_time": 1732359965, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276374, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:08.895964 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276375.sst) /dev/shm/ToplingTemp/Topling-160298-RreiCc 2024/11/23-19:06:10.208372 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276375.sst): first pass time = 1.31's, 102.274'MB/sec 2024/11/23-19:06:11.683047 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276375.sst): old prealloc_size = 147639500, real_size = 34025184 2024/11/23-19:06:11.683755 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276375.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929140 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0320 GB avg-val = 116.45 avg-zval = 34.46 2024/11/23-19:06:11.684577 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276375.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:12.673902 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276375.sst): second pass time = 1.47's, 73.423'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 38.175'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2598.320'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.16's, 237.338'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33217.456'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.423'MB/sec zip pipeline throughput = 73.423'MB/sec entries = 929140 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.45 avg-zval = 34.46 usrkeys = 929140 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.08 avg-zval = 34.46 TagRS{ kinds = RS_Key0_TagN bytes = 145280 } TagArray{ size = 746143 bytes = 2798048 } seq expand size = 5223001 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4452 value = 3.3799 dict = 1.00 all = 3.5484 } Zip/UnZip{ index = 0.0957 value = 0.2959 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 9.648138 GB avg = 0.164 KB (by entry num) total key len = 1.622925 GB avg = 0.028 KB total ukey len = 1.127952 GB avg = 0.016 KB total ukey num = 0.072054129 Billion total entry num = 0.058861858 Billion write speed all = 0.307444669 MB/sec (with seq num) write speed all = 0.294599907 MB/sec (without seq num) 2024/11/23-19:06:12.721972 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276375: 929140 keys, 37825240 bytes, temperature: kWarm 2024/11/23-19:06:12.722077 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359972722031, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276375, "file_size": 37825240, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 32013648, "index_size": 1779072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26015920, "raw_average_key_size": 28, "raw_value_size": 108201828, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929140, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359968, "oldest_key_time": 0, "file_creation_time": 1732359968, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276375, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:12.757816 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276376.sst) /dev/shm/ToplingTemp/Topling-160298-Kyyr8b 2024/11/23-19:06:14.003202 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276376.sst): first pass time = 1.25's, 107.775'MB/sec 2024/11/23-19:06:15.505217 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276376.sst): old prealloc_size = 147639500, real_size = 31153984 2024/11/23-19:06:15.505787 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276376.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0291 GB avg-val = 115.79 avg-zval = 31.22 2024/11/23-19:06:15.506618 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276376.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:16.362568 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276376.sst): second pass time = 1.50's, 72.057'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.277'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3240.268'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 270.873'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34918.049'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.057'MB/sec zip pipeline throughput = 72.057'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 31.22 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 31.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5138 value = 3.7084 dict = 1.00 all = 3.7927 } Zip/UnZip{ index = 0.0951 value = 0.2697 dict = 1.00 all = 0.2637 } ---------------------------- total value len = 9.762753 GB avg = 0.163 KB (by entry num) total key len = 1.649061 GB avg = 0.028 KB total ukey len = 1.146621 GB avg = 0.016 KB total ukey num = 0.072987586 Billion total entry num = 0.059795315 Billion write speed all = 0.311252694 MB/sec (with seq num) write speed all = 0.298205547 MB/sec (without seq num) 2024/11/23-19:06:16.408792 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276376: 933457 keys, 35388278 bytes, temperature: kWarm 2024/11/23-19:06:16.408852 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359976408820, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276376, "file_size": 35388278, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 29144752, "index_size": 1775680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359972, "oldest_key_time": 0, "file_creation_time": 1732359972, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276376, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:16.445323 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276377.sst) /dev/shm/ToplingTemp/Topling-160298-iWeJTa 2024/11/23-19:06:17.646576 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276377.sst): first pass time = 1.20's, 111.735'MB/sec 2024/11/23-19:06:19.199509 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276377.sst): old prealloc_size = 147639500, real_size = 33035600 2024/11/23-19:06:19.200120 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276377.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931033 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0310 GB avg-val = 116.16 avg-zval = 33.31 2024/11/23-19:06:19.201907 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276377.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:20.068474 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276377.sst): second pass time = 1.54's, 70.045'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.882'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3044.217'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 286.744'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34029.783'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.045'MB/sec zip pipeline throughput = 70.045'MB/sec entries = 931033 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.16 avg-zval = 33.31 usrkeys = 931033 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.16 avg-zval = 33.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931033 bytes = 3375008 } seq expand size = 6517231 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.3780 value = 3.4877 dict = 1.00 all = 3.6020 } Zip/UnZip{ index = 0.0964 value = 0.2867 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 9.877420 GB avg = 0.163 KB (by entry num) total key len = 1.675130 GB avg = 0.028 KB total ukey len = 1.165242 GB avg = 0.016 KB total ukey num = 0.073918619 Billion total entry num = 0.060726348 Billion write speed all = 0.315059342 MB/sec (with seq num) write speed all = 0.301810386 MB/sec (without seq num) 2024/11/23-19:06:20.116517 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276377: 931033 keys, 37262176 bytes, temperature: kWarm 2024/11/23-19:06:20.116587 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359980116552, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276377, "file_size": 37262176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776268, "table_properties": {"data_size": 31008448, "index_size": 1794240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26068924, "raw_average_key_size": 28, "raw_value_size": 108148938, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931033, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359976, "oldest_key_time": 0, "file_creation_time": 1732359976, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276377, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:20.154722 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276378.sst) /dev/shm/ToplingTemp/Topling-160298-dUJdla 2024/11/23-19:06:21.329744 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276378.sst): first pass time = 1.17's, 114.230'MB/sec 2024/11/23-19:06:22.869673 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276378.sst): old prealloc_size = 147639500, real_size = 34461392 2024/11/23-19:06:22.870267 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276378.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923457 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0325 GB avg-val = 117.34 avg-zval = 35.15 2024/11/23-19:06:22.872207 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276378.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:23.736822 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276378.sst): second pass time = 1.54's, 70.396'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 44.724'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3073.035'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.13's, 300.129'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 39657.044'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.396'MB/sec zip pipeline throughput = 70.396'MB/sec entries = 923457 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.34 avg-zval = 35.15 usrkeys = 923457 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.34 avg-zval = 35.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923457 bytes = 3462976 } seq expand size = 6464199 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.4418 value = 3.3381 dict = 1.00 all = 3.4608 } Zip/UnZip{ index = 0.0958 value = 0.2996 dict = 1.00 all = 0.2889 } ---------------------------- total value len = 9.992245 GB avg = 0.162 KB (by entry num) total key len = 1.700987 GB avg = 0.028 KB total ukey len = 1.183711 GB avg = 0.016 KB total ukey num = 0.074842076 Billion total entry num = 0.061649805 Billion write speed all = 0.318864100 MB/sec (with seq num) write speed all = 0.305415014 MB/sec (without seq num) 2024/11/23-19:06:23.784029 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276378: 923457 keys, 38781966 bytes, temperature: kWarm 2024/11/23-19:06:23.784098 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359983784063, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276378, "file_size": 38781966, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012774796, "table_properties": {"data_size": 32461568, "index_size": 1768768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25856796, "raw_average_key_size": 28, "raw_value_size": 108361014, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359980, "oldest_key_time": 0, "file_creation_time": 1732359980, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276378, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:23.822422 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276379.sst) /dev/shm/ToplingTemp/Topling-160298-s79gW9 2024/11/23-19:06:24.898312 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276379.sst): first pass time = 1.08's, 124.755'MB/sec 2024/11/23-19:06:26.510372 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276379.sst): old prealloc_size = 147639500, real_size = 32948704 2024/11/23-19:06:26.510931 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276379.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933794 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 115.73 avg-zval = 33.13 2024/11/23-19:06:26.511792 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276379.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:27.395211 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276379.sst): second pass time = 1.61's, 67.098'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.025'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3322.183'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 282.159'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33262.888'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.098'MB/sec zip pipeline throughput = 67.098'MB/sec entries = 933794 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.73 avg-zval = 33.13 usrkeys = 933794 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.73 avg-zval = 33.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933794 bytes = 3385024 } seq expand size = 6536558 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5146 value = 3.4931 dict = 1.00 all = 3.6095 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 10.106853 GB avg = 0.161 KB (by entry num) total key len = 1.727133 GB avg = 0.028 KB total ukey len = 1.202387 GB avg = 0.016 KB total ukey num = 0.075775870 Billion total entry num = 0.062583599 Billion write speed all = 0.322670158 MB/sec (with seq num) write speed all = 0.309018725 MB/sec (without seq num) 2024/11/23-19:06:27.440986 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276379: 933794 keys, 37184342 bytes, temperature: kWarm 2024/11/23-19:06:27.441079 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359987441032, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276379, "file_size": 37184342, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 30938896, "index_size": 1776192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26146232, "raw_average_key_size": 28, "raw_value_size": 108071628, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933794, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359983, "oldest_key_time": 0, "file_creation_time": 1732359983, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276379, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:27.496190 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276380.sst) /dev/shm/ToplingTemp/Topling-160298-RI4XY9 2024/11/23-19:06:28.671392 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276380.sst): first pass time = 1.18's, 114.215'MB/sec 2024/11/23-19:06:29.996871 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276380.sst): old prealloc_size = 147639500, real_size = 33343424 2024/11/23-19:06:29.997492 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276380.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924328 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.21 avg-zval = 33.92 2024/11/23-19:06:29.998345 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276380.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:30.885036 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276380.sst): second pass time = 1.32's, 81.813'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 42.411'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2936.158'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.13's, 285.889'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32005.798'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.813'MB/sec zip pipeline throughput = 81.813'MB/sec entries = 924328 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.21 avg-zval = 33.92 usrkeys = 924328 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.21 avg-zval = 33.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924328 bytes = 3466240 } seq expand size = 6470296 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5118 value = 3.4553 dict = 1.00 all = 3.5633 } Zip/UnZip{ index = 0.0951 value = 0.2894 dict = 1.00 all = 0.2806 } ---------------------------- total value len = 10.221660 GB avg = 0.161 KB (by entry num) total key len = 1.753014 GB avg = 0.028 KB total ukey len = 1.220873 GB avg = 0.016 KB total ukey num = 0.076700198 Billion total entry num = 0.063507927 Billion write speed all = 0.326475149 MB/sec (with seq num) write speed all = 0.312623409 MB/sec (without seq num) 2024/11/23-19:06:30.932122 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276380: 924328 keys, 37666388 bytes, temperature: kWarm 2024/11/23-19:06:30.932192 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359990932157, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276380, "file_size": 37666388, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 31353520, "index_size": 1758656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25881184, "raw_average_key_size": 28, "raw_value_size": 108336584, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924328, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359987, "oldest_key_time": 0, "file_creation_time": 1732359987, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276380, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:30.971173 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276381.sst) /dev/shm/ToplingTemp/Topling-160298-P9YCB9 2024/11/23-19:06:32.139054 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276381.sst): first pass time = 1.17's, 114.928'MB/sec 2024/11/23-19:06:33.450815 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276381.sst): old prealloc_size = 147639500, real_size = 33072752 2024/11/23-19:06:33.451423 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276381.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926900 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.80 avg-zval = 33.50 2024/11/23-19:06:33.452292 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276381.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:34.299805 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276381.sst): second pass time = 1.31's, 82.580'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.056'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3057.628'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 294.453'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35771.447'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.580'MB/sec zip pipeline throughput = 82.580'MB/sec entries = 926900 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.80 avg-zval = 33.50 usrkeys = 926900 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.80 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926900 bytes = 3475888 } seq expand size = 6488300 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3760 value = 3.4863 dict = 1.00 all = 3.5885 } Zip/UnZip{ index = 0.0964 value = 0.2868 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 10.336413 GB avg = 0.160 KB (by entry num) total key len = 1.778968 GB avg = 0.028 KB total ukey len = 1.239411 GB avg = 0.016 KB total ukey num = 0.077627098 Billion total entry num = 0.064434827 Billion write speed all = 0.330280582 MB/sec (with seq num) write speed all = 0.316227984 MB/sec (without seq num) 2024/11/23-19:06:34.345329 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276381: 926900 keys, 37402580 bytes, temperature: kWarm 2024/11/23-19:06:34.345402 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359994345366, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276381, "file_size": 37402580, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 31054240, "index_size": 1786624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953200, "raw_average_key_size": 28, "raw_value_size": 108264564, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926900, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359990, "oldest_key_time": 0, "file_creation_time": 1732359990, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276381, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:34.381794 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276382.sst) /dev/shm/ToplingTemp/Topling-160298-ysh8ua 2024/11/23-19:06:35.385155 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276382.sst): first pass time = 1.00's, 133.773'MB/sec 2024/11/23-19:06:36.826190 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276382.sst): old prealloc_size = 147639500, real_size = 35704336 2024/11/23-19:06:36.826753 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276382.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929908 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 116.33 avg-zval = 36.24 2024/11/23-19:06:36.827142 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276382.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:37.577441 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276382.sst): second pass time = 1.44's, 75.116'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.75's, 50.342'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3266.780'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.372'MB/sec write SST data time = 0.13's, 301.206'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30520.657'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.116'MB/sec zip pipeline throughput = 75.116'MB/sec entries = 929908 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.33 avg-zval = 36.24 usrkeys = 929908 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.91 avg-zval = 36.24 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 341975 bytes = 1111440 } seq expand size = 2393825 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5121 value = 3.2099 dict = 1.00 all = 3.5490 } Zip/UnZip{ index = 0.0951 value = 0.3115 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 10.446987 GB avg = 0.160 KB (by entry num) total key len = 1.805005 GB avg = 0.028 KB total ukey len = 1.258009 GB avg = 0.016 KB total ukey num = 0.078557006 Billion total entry num = 0.065364735 Billion write speed all = 0.333974947 MB/sec (with seq num) write speed all = 0.319720818 MB/sec (without seq num) 2024/11/23-19:06:37.624282 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276382: 929908 keys, 37818517 bytes, temperature: kWarm 2024/11/23-19:06:37.624357 139957685155392 EVENT_LOG_v1 {"time_micros": 1732359997624320, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276382, "file_size": 37818517, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 33702464, "index_size": 1769216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26037424, "raw_average_key_size": 28, "raw_value_size": 108180384, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929908, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359994, "oldest_key_time": 0, "file_creation_time": 1732359994, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276382, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:37.668479 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276383.sst) /dev/shm/ToplingTemp/Topling-160298-XQ3Zk9 2024/11/23-19:06:38.634104 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276383.sst): first pass time = 0.97's, 139.001'MB/sec 2024/11/23-19:06:40.117853 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276383.sst): old prealloc_size = 147639500, real_size = 37249392 2024/11/23-19:06:40.118427 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276383.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926204 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0352 GB avg-val = 116.91 avg-zval = 38.05 2024/11/23-19:06:40.118738 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276383.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:40.834339 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276383.sst): second pass time = 1.48's, 73.021'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 54.503'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3221.435'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.392'MB/sec write SST data time = 0.12's, 312.739'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31954.752'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.021'MB/sec zip pipeline throughput = 73.021'MB/sec entries = 926204 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.91 avg-zval = 38.05 usrkeys = 926204 avg-key = 20.00 avg-zkey = 1.91 avg-val = 118.52 avg-zval = 38.05 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 212544 bytes = 797056 } seq expand size = 1487808 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4468 value = 3.0724 dict = 1.00 all = 3.4371 } Zip/UnZip{ index = 0.0957 value = 0.3255 dict = 1.00 all = 0.2909 } ---------------------------- total value len = 10.556759 GB avg = 0.159 KB (by entry num) total key len = 1.830939 GB avg = 0.028 KB total ukey len = 1.276534 GB avg = 0.016 KB total ukey num = 0.079483210 Billion total entry num = 0.066290939 Billion write speed all = 0.337644147 MB/sec (with seq num) write speed all = 0.323189323 MB/sec (without seq num) 2024/11/23-19:06:40.882179 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276383: 926204 keys, 39049685 bytes, temperature: kWarm 2024/11/23-19:06:40.882252 139957685155392 EVENT_LOG_v1 {"time_micros": 1732360000882215, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276383, "file_size": 39049685, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4024098993, "table_properties": {"data_size": 35244448, "index_size": 1773184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933712, "raw_average_key_size": 28, "raw_value_size": 108284080, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926204, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732359997, "oldest_key_time": 0, "file_creation_time": 1732359997, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276383, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:40.924034 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276384.sst) /dev/shm/ToplingTemp/Topling-160298-iTRPCb 2024/11/23-19:06:41.803940 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276384.sst): first pass time = 0.88's, 152.544'MB/sec 2024/11/23-19:06:43.170119 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276384.sst): old prealloc_size = 147639500, real_size = 35437136 2024/11/23-19:06:43.170649 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276384.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927373 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0334 GB avg-val = 116.73 avg-zval = 36.05 2024/11/23-19:06:43.171459 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276384.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:44.001465 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276384.sst): second pass time = 1.37's, 79.285'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.560'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3502.943'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 315.340'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36137.606'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.285'MB/sec zip pipeline throughput = 79.285'MB/sec entries = 927373 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.73 avg-zval = 36.05 usrkeys = 927373 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.73 avg-zval = 36.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927373 bytes = 3245824 } seq expand size = 6491611 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4438 value = 3.2382 dict = 1.00 all = 3.3947 } Zip/UnZip{ index = 0.0958 value = 0.3088 dict = 1.00 all = 0.2946 } ---------------------------- total value len = 10.671502 GB avg = 0.159 KB (by entry num) total key len = 1.856905 GB avg = 0.028 KB total ukey len = 1.295081 GB avg = 0.016 KB total ukey num = 0.080410583 Billion total entry num = 0.067218312 Billion write speed all = 0.341449903 MB/sec (with seq num) write speed all = 0.326794130 MB/sec (without seq num) 2024/11/23-19:06:44.049344 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276384: 927373 keys, 39537446 bytes, temperature: kWarm 2024/11/23-19:06:44.049414 139957685155392 EVENT_LOG_v1 {"time_micros": 1732360004049377, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276384, "file_size": 39537446, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4024098994, "largest_seqno": 4204201741, "table_properties": {"data_size": 33429184, "index_size": 1775936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966444, "raw_average_key_size": 28, "raw_value_size": 108251294, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927373, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360000, "oldest_key_time": 0, "file_creation_time": 1732360000, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276384, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:44.086929 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276385.sst) /dev/shm/ToplingTemp/Topling-160298-VHmbgc 2024/11/23-19:06:44.931234 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276385.sst): first pass time = 0.84's, 158.975'MB/sec 2024/11/23-19:06:46.255783 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276385.sst): old prealloc_size = 147639500, real_size = 34053808 2024/11/23-19:06:46.256404 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276385.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926280 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0321 GB avg-val = 116.90 avg-zval = 34.61 2024/11/23-19:06:46.257183 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276385.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:06:47.107228 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276385.sst): second pass time = 1.32's, 81.797'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.536'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2944.301'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 294.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35909.145'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.797'MB/sec zip pipeline throughput = 81.797'MB/sec entries = 926280 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.90 avg-zval = 34.61 usrkeys = 926280 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.90 avg-zval = 34.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926280 bytes = 3010432 } seq expand size = 6483960 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5129 value = 3.3775 dict = 1.00 all = 3.5396 } Zip/UnZip{ index = 0.0951 value = 0.2961 dict = 1.00 all = 0.2825 } ---------------------------- total value len = 10.786268 GB avg = 0.158 KB (by entry num) total key len = 1.882841 GB avg = 0.028 KB total ukey len = 1.313607 GB avg = 0.016 KB total ukey num = 0.081336863 Billion total entry num = 0.068144592 Billion write speed all = 0.345255373 MB/sec (with seq num) write speed all = 0.330398899 MB/sec (without seq num) 2024/11/23-19:06:47.155083 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276385: 926280 keys, 37919122 bytes, temperature: kWarm 2024/11/23-19:06:47.155156 139957685155392 EVENT_LOG_v1 {"time_micros": 1732360007155118, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276385, "file_size": 37919122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4141699119, "largest_seqno": 4204201900, "table_properties": {"data_size": 32059872, "index_size": 1762176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935840, "raw_average_key_size": 28, "raw_value_size": 108281996, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926280, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360004, "oldest_key_time": 0, "file_creation_time": 1732360004, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276385, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:47.192922 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276386.sst) /dev/shm/ToplingTemp/Topling-160298-ijWNBd 2024/11/23-19:06:47.319581 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276386.sst): first pass time = 0.13's, 234.153'MB/sec 2024/11/23-19:06:47.593197 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276386.sst): old prealloc_size = 147639500, real_size = 8234656 2024/11/23-19:06:47.593372 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276386.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 205196 prefix = 4 raw-key = 0.0033 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0239 GB zip-val = 0.0078 GB avg-val = 116.48 avg-zval = 37.96 2024/11/23-19:06:47.593573 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276386.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.030 GB 2024/11/23-19:06:47.778499 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00/276386.sst): second pass time = 0.27's, 87.617'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 48.979'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2620.938'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.329'MB/sec write SST data time = 0.03's, 311.464'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.249'MB, throughput = 8607.001'MB/sec zip my value time = 0.27's, unzip length = 0.024'GB zip my value throughput = 87.617'MB/sec zip pipeline throughput = 87.617'MB/sec entries = 205196 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.48 avg-zval = 37.96 usrkeys = 205196 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.48 avg-zval = 37.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 205196 bytes = 641248 } seq expand size = 1436372 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0041 GB value = 0.0239 GB dict = 0.25 MB all = 0.0296 GB } __ZipSize{ index = 0.0004 GB value = 0.0078 GB dict = 0.25 MB all = 0.0091 GB } UnZip/Zip{ index = 10.4114 value = 3.0685 dict = 1.00 all = 3.2669 } Zip/UnZip{ index = 0.0960 value = 0.3259 dict = 1.00 all = 0.3061 } ---------------------------- total value len = 10.811605 GB avg = 0.158 KB (by entry num) total key len = 1.888587 GB avg = 0.028 KB total ukey len = 1.317710 GB avg = 0.016 KB total ukey num = 0.081542059 Billion total entry num = 0.068349788 Billion write speed all = 0.346096097 MB/sec (with seq num) write speed all = 0.331195160 MB/sec (without seq num) 2024/11/23-19:06:47.795718 139957685155392 [compaction_job.cc:1948] [default] [JOB 33807] Generated table #276386: 205196 keys, 9074787 bytes, temperature: kWarm 2024/11/23-19:06:47.795787 139957685155392 EVENT_LOG_v1 {"time_micros": 1732360007795751, "cf_name": "default", "job": 33807, "event": "table_file_creation", "file_number": 276386, "file_size": 9074787, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4180272482, "largest_seqno": 4202706189, "table_properties": {"data_size": 7789008, "index_size": 394176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5745488, "raw_average_key_size": 28, "raw_value_size": 23900744, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 205196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360007, "oldest_key_time": 0, "file_creation_time": 1732360007, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276386, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:06:47.812413 139957685155392 [compaction_job.cc:703] job-33807: subcompact[0], size: 0.660023 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33807/att-00 [276369,276370,276371,276372,276373,276374,276375,276376,276377,276378,276379,276380,276381,276382,276383,276384,276385,276386] 2024/11/23-19:06:47.923939 139957685155392 (Original Log Time 2024/11/23-19:05:45.209191) EVENT_LOG_v1 {"time_micros": 1732359945209150, "job": 33807, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [276348], "files_L3": [276365, 276366, 276290, 276291, 276292, 276294, 276295, 276296, 276297, 276298, 276299, 276300, 276301, 276302, 276303, 276304, 276317, 276318], "score": 0.989842, "input_data_size": 679592153, "oldest_snapshot_seqno": 4204283361} 2024/11/23-19:06:47.923943 139957685155392 (Original Log Time 2024/11/23-19:06:47.922450) [compaction_job.cc:2022] [default] [JOB 33807] Compacted 1@2 + 18@3 files to L3 => 660023419 bytes 2024/11/23-19:06:47.923944 139957685155392 (Original Log Time 2024/11/23-19:06:47.923873) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 18 0 0 0] max score 0.02, MB/sec: 10.9 rd, 10.5 wr, level 3, files in(1, 18) out(18 +0 blob) MB in(27.2, 620.9 +0.0 blob) out(629.4 +0.0 blob), read-write-amplify(47.0) write-amplify(23.2) OK, records in: 16061128, records dropped: 69504 output_compression: Snappy 2024/11/23-19:06:47.923950 139957685155392 (Original Log Time 2024/11/23-19:06:47.923916) EVENT_LOG_v1 {"time_micros": 1732360007923896, "cf": "default", "job": 33807, "event": "compaction_finished", "compaction_time_micros": 62603228, "compaction_time_cpu_micros": 26215855, "output_level": 3, "num_output_files": 18, "total_output_size": 660023419, "num_input_records": 16061128, "num_output_records": 15991624, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 497236300, "file_range_sync_nanos": 0, "file_fsync_nanos": 14059, "file_prepare_write_nanos": 56707, "lsm_state": [0, 0, 0, 18, 0, 0, 0]} 2024/11/23-19:06:48.057302 139957685155392 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732359945,"executesMs":62847,"compactionJobId":33807,"attempt":0,"compactionInputRawBytes":2321218432,"compactionInputZipBytes":679592153,"compactionOutputRawBytes":0,"compactionOutputZipBytes":660023419,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-a1f8ce340e229be3e3283d19a99310df-2f39c5247b98588a-00"}}, response = 2024/11/23-19:06:48.057393 139957685155392 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33807/att-00: olev 3, work 62.847 s, result 7.701 ms, install 1.516 ms, input{raw 2.162 GiB zip 648.110 MiB}