2024/11/23-05:48:19.676007 140024777004608 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00: fork to child time = 0.009256 sec 2024/11/23-05:48:19.676811 140024777004608 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00 2024/11/23-05:48:19.684253 140024777004608 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00 2024/11/23-05:48:19.685365 140024777004608 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-05:48:19.685638 140024777004608 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-05:48:19.731095 140024777004608 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20478/att-00/MANIFEST-206347 2024/11/23-05:48:19.746996 140024777004608 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20478/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 247282, last_sequence is 3819332534, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 247125 2024/11/23-05:48:19.747004 140024777004608 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-05:48:19.747123 140024777004608 [version_set.cc:5721] Creating manifest 247282 2024/11/23-05:48:19.777219 140024777004608 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-05:48:19.796277 140024777004608 [compaction_job.cc:2437] [default] [JOB 20478] Compacting 1@2 + 16@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-05:48:19.796298 140024777004608 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 15310 Base level 2, inputs: [247274(27MB)], [247219(34MB) 247220(35MB) 247221(34MB) 247222(875KB) 247186(35MB) 247187(35MB) 247188(35MB) 247189(36MB) 247190(34MB) 247191(36MB) 247192(35MB) 247193(37MB) 247194(35MB) 247195(35MB) 247196(35MB) 247197(35MB)] 2024/11/23-05:48:19.802687 140024777004608 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-05:48:19.802718 140024777004608 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-05:48:19.802737 140024777004608 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-05:48:19.802742 140024777004608 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-05:48:19.802753 140024777004608 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-05:48:20.347862 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247283.sst) /dev/shm/ToplingTemp/Topling-426019-c8xW4K 2024/11/23-05:48:21.116332 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247283.sst): first pass time = 0.77's, 174.666'MB/sec 2024/11/23-05:48:23.205040 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247283.sst): old prealloc_size = 147639500, real_size = 32593424 2024/11/23-05:48:23.205737 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247283.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927649 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.69 avg-zval = 32.98 2024/11/23-05:48:23.206574 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247283.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:48:24.548926 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247283.sst): second pass time = 2.09's, 51.847'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.34's, 27.219'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2599.552'MB/sec (index lex order gen) rebuild zvType time = 1.17's, 0.199'MB/sec write SST data time = 0.18's, 206.776'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35726.115'MB/sec zip my value time = 2.09's, unzip length = 0.108'GB zip my value throughput = 51.847'MB/sec zip pipeline throughput = 65.190'MB/sec entries = 927649 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.69 avg-zval = 32.98 usrkeys = 927649 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.69 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927649 bytes = 3130832 } seq expand size = 6493543 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5105 value = 3.5378 dict = 1.00 all = 3.6693 } Zip/UnZip{ index = 0.0951 value = 0.2827 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 0.114737 GB avg = 0.124 KB (by entry num) total key len = 0.025974 GB avg = 0.028 KB total ukey len = 0.018553 GB avg = 0.020 KB total ukey num = 0.000927649 Billion total entry num = 0.000927649 Billion write speed all = 0.001867153 MB/sec (with seq num) write speed all = 0.001768679 MB/sec (without seq num) 2024/11/23-05:48:24.581503 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247283: 927649 keys, 36578514 bytes, temperature: kWarm 2024/11/23-05:48:24.581582 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312104581532, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247283, "file_size": 36578514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3726348054, "largest_seqno": 3819299302, "table_properties": {"data_size": 30596160, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25974172, "raw_average_key_size": 28, "raw_value_size": 108243670, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927649, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312100, "oldest_key_time": 0, "file_creation_time": 1732312100, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247283, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:48:24.617274 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247284.sst) /dev/shm/ToplingTemp/Topling-426019-wiBRqJ 2024/11/23-05:48:25.586519 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247284.sst): first pass time = 0.97's, 138.482'MB/sec 2024/11/23-05:48:27.273587 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247284.sst): old prealloc_size = 147639500, real_size = 33212240 2024/11/23-05:48:27.274284 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247284.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927826 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.66 avg-zval = 33.63 2024/11/23-05:48:27.275141 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247284.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:48:28.720029 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247284.sst): second pass time = 1.69's, 64.187'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.45's, 25.877'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2636.239'MB/sec (index lex order gen) rebuild zvType time = 1.29's, 0.180'MB/sec write SST data time = 0.16's, 233.287'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33686.732'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.187'MB/sec zip pipeline throughput = 64.187'MB/sec entries = 927826 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.66 avg-zval = 33.63 usrkeys = 927826 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.66 avg-zval = 33.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927826 bytes = 3363392 } seq expand size = 6494782 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4459 value = 3.4688 dict = 1.00 all = 3.5859 } Zip/UnZip{ index = 0.0957 value = 0.2883 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 0.229471 GB avg = 0.124 KB (by entry num) total key len = 0.051953 GB avg = 0.028 KB total ukey len = 0.037109 GB avg = 0.020 KB total ukey num = 0.001855475 Billion total entry num = 0.001855475 Billion write speed all = 0.003734116 MB/sec (with seq num) write speed all = 0.003537159 MB/sec (without seq num) 2024/11/23-05:48:28.751088 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247284: 927826 keys, 37429456 bytes, temperature: kWarm 2024/11/23-05:48:28.751153 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312108751120, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247284, "file_size": 37429456, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3819299313, "table_properties": {"data_size": 31203648, "index_size": 1776448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979128, "raw_average_key_size": 28, "raw_value_size": 108238672, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927826, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312104, "oldest_key_time": 0, "file_creation_time": 1732312104, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247284, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:48:28.787559 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247285.sst) /dev/shm/ToplingTemp/Topling-426019-lAnleL 2024/11/23-05:48:29.636520 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247285.sst): first pass time = 0.85's, 158.103'MB/sec 2024/11/23-05:48:31.611807 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247285.sst): old prealloc_size = 147639500, real_size = 32594080 2024/11/23-05:48:31.612555 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247285.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927905 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.65 avg-zval = 32.96 2024/11/23-05:48:31.613355 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247285.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:48:32.694736 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247285.sst): second pass time = 1.97's, 54.844'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 33.778'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2452.803'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.245'MB/sec write SST data time = 0.13's, 273.897'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32992.699'MB/sec zip my value time = 1.97's, unzip length = 0.108'GB zip my value throughput = 54.844'MB/sec zip pipeline throughput = 63.466'MB/sec entries = 927905 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.65 avg-zval = 32.96 usrkeys = 927905 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.65 avg-zval = 32.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927905 bytes = 3131696 } seq expand size = 6495335 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4441 value = 3.5389 dict = 1.00 all = 3.6692 } Zip/UnZip{ index = 0.0957 value = 0.2826 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 0.344203 GB avg = 0.124 KB (by entry num) total key len = 0.077935 GB avg = 0.028 KB total ukey len = 0.055668 GB avg = 0.020 KB total ukey num = 0.002783380 Billion total entry num = 0.002783380 Billion write speed all = 0.005600894 MB/sec (with seq num) write speed all = 0.005305456 MB/sec (without seq num) 2024/11/23-05:48:32.728307 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247285: 927905 keys, 36579616 bytes, temperature: kWarm 2024/11/23-05:48:32.728415 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312112728339, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247285, "file_size": 36579616, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3712241388, "largest_seqno": 3819299828, "table_properties": {"data_size": 30585040, "index_size": 1776896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981340, "raw_average_key_size": 28, "raw_value_size": 108236522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312108, "oldest_key_time": 0, "file_creation_time": 1732312108, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247285, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:48:32.766142 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247286.sst) /dev/shm/ToplingTemp/Topling-426019-ZC6FdK 2024/11/23-05:48:33.793883 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247286.sst): first pass time = 1.03's, 130.600'MB/sec 2024/11/23-05:48:35.268081 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247286.sst): old prealloc_size = 147639500, real_size = 33273968 2024/11/23-05:48:35.268735 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247286.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927882 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.65 avg-zval = 33.70 2024/11/23-05:48:35.269621 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247286.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:48:36.326044 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247286.sst): second pass time = 1.47's, 73.495'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 35.438'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2800.084'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.16's, 239.451'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31819.210'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.495'MB/sec zip pipeline throughput = 73.495'MB/sec entries = 927882 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.65 avg-zval = 33.70 usrkeys = 927882 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.65 avg-zval = 33.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927882 bytes = 3363584 } seq expand size = 6495174 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4450 value = 3.4619 dict = 1.00 all = 3.5800 } Zip/UnZip{ index = 0.0957 value = 0.2889 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 0.458935 GB avg = 0.124 KB (by entry num) total key len = 0.103915 GB avg = 0.028 KB total ukey len = 0.074225 GB avg = 0.020 KB total ukey num = 0.003711262 Billion total entry num = 0.003711262 Billion write speed all = 0.007467506 MB/sec (with seq num) write speed all = 0.007073599 MB/sec (without seq num) 2024/11/23-05:48:36.359841 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247286: 927882 keys, 37491288 bytes, temperature: kWarm 2024/11/23-05:48:36.359905 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312116359871, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247286, "file_size": 37491288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3819299343, "table_properties": {"data_size": 31265120, "index_size": 1776704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980696, "raw_average_key_size": 28, "raw_value_size": 108237044, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927882, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312112, "oldest_key_time": 0, "file_creation_time": 1732312112, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247286, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:48:36.395362 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247287.sst) /dev/shm/ToplingTemp/Topling-426019-igmjUJ 2024/11/23-05:48:37.294751 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247287.sst): first pass time = 0.90's, 149.238'MB/sec 2024/11/23-05:48:38.734866 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247287.sst): old prealloc_size = 147639500, real_size = 32914304 2024/11/23-05:48:38.735559 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247287.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928660 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.53 avg-zval = 33.27 2024/11/23-05:48:38.736362 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247287.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:48:39.783507 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247287.sst): second pass time = 1.44's, 75.230'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.080'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2655.117'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.261'MB/sec write SST data time = 0.16's, 231.540'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26387.920'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.230'MB/sec zip pipeline throughput = 75.230'MB/sec entries = 928660 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.53 avg-zval = 33.27 usrkeys = 928660 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.53 avg-zval = 33.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928660 bytes = 3018160 } seq expand size = 6500620 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.3790 value = 3.5030 dict = 1.00 all = 3.6486 } Zip/UnZip{ index = 0.0963 value = 0.2855 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 0.573651 GB avg = 0.124 KB (by entry num) total key len = 0.129918 GB avg = 0.028 KB total ukey len = 0.092798 GB avg = 0.020 KB total ukey num = 0.004639922 Billion total entry num = 0.004639922 Billion write speed all = 0.009334033 MB/sec (with seq num) write speed all = 0.008841581 MB/sec (without seq num) 2024/11/23-05:48:39.814977 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247287: 928660 keys, 36786080 bytes, temperature: kWarm 2024/11/23-05:48:39.815036 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312119815005, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247287, "file_size": 36786080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759264366, "largest_seqno": 3819299374, "table_properties": {"data_size": 30892464, "index_size": 1789504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002480, "raw_average_key_size": 28, "raw_value_size": 108215284, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928660, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312116, "oldest_key_time": 0, "file_creation_time": 1732312116, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247287, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:48:39.862097 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247288.sst) /dev/shm/ToplingTemp/Topling-426019-hAbE5K 2024/11/23-05:48:40.858419 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247288.sst): first pass time = 1.00's, 134.719'MB/sec 2024/11/23-05:48:42.533860 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247288.sst): old prealloc_size = 147639500, real_size = 33079152 2024/11/23-05:48:42.534450 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247288.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926777 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.82 avg-zval = 33.53 2024/11/23-05:48:42.535284 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247288.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:48:43.487393 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247288.sst): second pass time = 1.67's, 64.647'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.111'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3111.297'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 285.571'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28164.659'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 64.647'MB/sec zip pipeline throughput = 127.010'MB/sec entries = 926777 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.82 avg-zval = 33.53 usrkeys = 926777 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.82 avg-zval = 33.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926777 bytes = 3359584 } seq expand size = 6487439 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4446 value = 3.4844 dict = 1.00 all = 3.5990 } Zip/UnZip{ index = 0.0957 value = 0.2870 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 0.688406 GB avg = 0.124 KB (by entry num) total key len = 0.155868 GB avg = 0.028 KB total ukey len = 0.111334 GB avg = 0.020 KB total ukey num = 0.005566699 Billion total entry num = 0.005566699 Billion write speed all = 0.011200176 MB/sec (with seq num) write speed all = 0.010609391 MB/sec (without seq num) 2024/11/23-05:48:43.521298 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247288: 926777 keys, 37292946 bytes, temperature: kWarm 2024/11/23-05:48:43.521362 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312123521328, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247288, "file_size": 37292946, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3819296957, "table_properties": {"data_size": 31072608, "index_size": 1774656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949756, "raw_average_key_size": 28, "raw_value_size": 108267986, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926777, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312119, "oldest_key_time": 0, "file_creation_time": 1732312119, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247288, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:48:43.555619 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247289.sst) /dev/shm/ToplingTemp/Topling-426019-WRO2gL 2024/11/23-05:48:44.365965 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247289.sst): first pass time = 0.81's, 165.637'MB/sec 2024/11/23-05:48:45.836908 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247289.sst): old prealloc_size = 147639500, real_size = 34082448 2024/11/23-05:48:45.837581 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247289.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928528 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0321 GB avg-val = 116.55 avg-zval = 34.55 2024/11/23-05:48:45.838462 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247289.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:48:46.986797 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247289.sst): second pass time = 1.47's, 73.613'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.15's, 33.310'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2708.195'MB/sec (index lex order gen) rebuild zvType time = 0.99's, 0.234'MB/sec write SST data time = 0.16's, 240.857'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 39884.013'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.613'MB/sec zip pipeline throughput = 73.613'MB/sec entries = 928528 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 34.55 usrkeys = 928528 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 34.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928528 bytes = 3365936 } seq expand size = 6499696 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5147 value = 3.3730 dict = 1.00 all = 3.5042 } Zip/UnZip{ index = 0.0951 value = 0.2965 dict = 1.00 all = 0.2854 } ---------------------------- total value len = 0.803125 GB avg = 0.124 KB (by entry num) total key len = 0.181866 GB avg = 0.028 KB total ukey len = 0.129905 GB avg = 0.020 KB total ukey num = 0.006495227 Billion total entry num = 0.006495227 Billion write speed all = 0.013066335 MB/sec (with seq num) write speed all = 0.012377039 MB/sec (without seq num) 2024/11/23-05:48:47.025062 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247289: 928528 keys, 38301772 bytes, temperature: kWarm 2024/11/23-05:48:47.025123 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312127025093, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247289, "file_size": 38301772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3819299479, "table_properties": {"data_size": 32083968, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25998784, "raw_average_key_size": 28, "raw_value_size": 108219072, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928528, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312123, "oldest_key_time": 0, "file_creation_time": 1732312123, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247289, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:48:47.063139 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247290.sst) /dev/shm/ToplingTemp/Topling-426019-JMWBeL 2024/11/23-05:48:48.054876 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247290.sst): first pass time = 0.99's, 135.342'MB/sec 2024/11/23-05:48:49.833659 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247290.sst): old prealloc_size = 147639500, real_size = 31736352 2024/11/23-05:48:49.834253 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247290.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927831 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0297 GB avg-val = 116.66 avg-zval = 32.05 2024/11/23-05:48:49.835034 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247290.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:48:50.985265 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247290.sst): second pass time = 1.78's, 60.872'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.15's, 31.020'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3082.766'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.24's, 149.097'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 21508.822'MB/sec zip my value time = 1.78's, unzip length = 0.108'GB zip my value throughput = 60.872'MB/sec zip pipeline throughput = 130.783'MB/sec entries = 927831 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 32.05 usrkeys = 927831 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 32.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927831 bytes = 3131440 } seq expand size = 6494817 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5133 value = 3.6396 dict = 1.00 all = 3.7573 } Zip/UnZip{ index = 0.0951 value = 0.2748 dict = 1.00 all = 0.2661 } ---------------------------- total value len = 0.917858 GB avg = 0.124 KB (by entry num) total key len = 0.207846 GB avg = 0.028 KB total ukey len = 0.148461 GB avg = 0.020 KB total ukey num = 0.007423058 Billion total entry num = 0.007423058 Billion write speed all = 0.014932158 MB/sec (with seq num) write speed all = 0.014144439 MB/sec (without seq num) 2024/11/23-05:48:51.021480 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247290: 927831 keys, 35721660 bytes, temperature: kWarm 2024/11/23-05:48:51.021558 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312131021515, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247290, "file_size": 35721660, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3716868816, "largest_seqno": 3819299420, "table_properties": {"data_size": 29739152, "index_size": 1765056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979268, "raw_average_key_size": 28, "raw_value_size": 108238582, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927831, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312127, "oldest_key_time": 0, "file_creation_time": 1732312127, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247290, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:48:51.069389 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247291.sst) /dev/shm/ToplingTemp/Topling-426019-POsFwM 2024/11/23-05:48:51.938529 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247291.sst): first pass time = 0.87's, 154.435'MB/sec 2024/11/23-05:48:53.528364 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247291.sst): old prealloc_size = 147639500, real_size = 34364304 2024/11/23-05:48:53.528926 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247291.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930926 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.18 avg-zval = 34.74 2024/11/23-05:48:53.529518 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247291.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:48:54.660608 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247291.sst): second pass time = 1.59's, 68.058'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 33.147'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3326.770'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.264'MB/sec write SST data time = 0.25's, 149.831'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34293.237'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.058'MB/sec zip pipeline throughput = 68.058'MB/sec entries = 930926 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.18 avg-zval = 34.74 usrkeys = 930926 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.52 avg-zval = 34.74 TagRS{ kinds = RS_Key0_TagN bytes = 145560 } TagArray{ size = 577411 bytes = 2165312 } seq expand size = 4041877 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3768 value = 3.3445 dict = 1.00 all = 3.5762 } Zip/UnZip{ index = 0.0964 value = 0.2990 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 1.030052 GB avg = 0.123 KB (by entry num) total key len = 0.233912 GB avg = 0.028 KB total ukey len = 0.167080 GB avg = 0.020 KB total ukey num = 0.008353984 Billion total entry num = 0.008353984 Billion write speed all = 0.016765318 MB/sec (with seq num) write speed all = 0.015878855 MB/sec (without seq num) 2024/11/23-05:48:54.694122 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247291: 930926 keys, 37530344 bytes, temperature: kWarm 2024/11/23-05:48:54.694190 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312134694154, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247291, "file_size": 37530344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3819299562, "table_properties": {"data_size": 32337152, "index_size": 1794240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26065928, "raw_average_key_size": 28, "raw_value_size": 108151908, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930926, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312131, "oldest_key_time": 0, "file_creation_time": 1732312131, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247291, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:48:54.740111 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247292.sst) /dev/shm/ToplingTemp/Topling-426019-lSsYJM 2024/11/23-05:48:55.484037 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247292.sst): first pass time = 0.74's, 180.426'MB/sec 2024/11/23-05:48:57.318136 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247292.sst): old prealloc_size = 147639500, real_size = 36456288 2024/11/23-05:48:57.318720 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247292.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927076 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.17 2024/11/23-05:48:57.318763 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247292.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:48:57.992586 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247292.sst): second pass time = 1.83's, 59.047'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.67's, 55.329'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3153.722'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.449'MB/sec write SST data time = 0.16's, 236.603'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 25103.104'MB/sec zip my value time = 1.83's, unzip length = 0.108'GB zip my value throughput = 59.047'MB/sec zip pipeline throughput = 59.047'MB/sec entries = 927076 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 37.17 usrkeys = 927076 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 37.17 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5124 value = 3.1416 dict = 1.00 all = 3.5969 } Zip/UnZip{ index = 0.0951 value = 0.3183 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 1.138312 GB avg = 0.123 KB (by entry num) total key len = 0.259870 GB avg = 0.028 KB total ukey len = 0.185621 GB avg = 0.020 KB total ukey num = 0.009281060 Billion total entry num = 0.009281060 Billion write speed all = 0.018544774 MB/sec (with seq num) write speed all = 0.017559979 MB/sec (without seq num) 2024/11/23-05:48:58.027391 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247292: 927076 keys, 37314524 bytes, temperature: kWarm 2024/11/23-05:48:58.027472 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312138027428, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247292, "file_size": 37314524, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34460560, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958128, "raw_average_key_size": 28, "raw_value_size": 108259612, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927076, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312134, "oldest_key_time": 0, "file_creation_time": 1732312134, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247292, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:48:58.078306 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247293.sst) /dev/shm/ToplingTemp/Topling-426019-r3U88I 2024/11/23-05:48:58.962586 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247293.sst): first pass time = 0.88's, 151.790'MB/sec 2024/11/23-05:49:00.369877 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247293.sst): old prealloc_size = 147639500, real_size = 36109648 2024/11/23-05:49:00.370531 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247293.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925395 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0341 GB avg-val = 117.04 avg-zval = 36.87 2024/11/23-05:49:00.371168 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247293.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:49:01.191954 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247293.sst): second pass time = 1.41's, 77.004'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 47.776'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2807.006'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.344'MB/sec write SST data time = 0.15's, 262.125'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32028.872'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 77.004'MB/sec zip pipeline throughput = 77.004'MB/sec entries = 925395 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 36.87 usrkeys = 925395 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.40 avg-zval = 36.87 TagRS{ kinds = RS_Key0_TagN bytes = 144680 } TagArray{ size = 576495 bytes = 2161872 } seq expand size = 4035465 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5109 value = 3.1745 dict = 1.00 all = 3.4174 } Zip/UnZip{ index = 0.0951 value = 0.3150 dict = 1.00 all = 0.2926 } ---------------------------- total value len = 1.250654 GB avg = 0.123 KB (by entry num) total key len = 0.285781 GB avg = 0.028 KB total ukey len = 0.204129 GB avg = 0.020 KB total ukey num = 0.010206455 Billion total entry num = 0.010206455 Billion write speed all = 0.020377632 MB/sec (with seq num) write speed all = 0.019294691 MB/sec (without seq num) 2024/11/23-05:49:01.227452 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247293: 925395 keys, 39274942 bytes, temperature: kWarm 2024/11/23-05:49:01.227536 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312141227493, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247293, "file_size": 39274942, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3819299163, "table_properties": {"data_size": 34117312, "index_size": 1760832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911060, "raw_average_key_size": 28, "raw_value_size": 108306786, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925395, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312138, "oldest_key_time": 0, "file_creation_time": 1732312138, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247293, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:49:01.274979 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247294.sst) /dev/shm/ToplingTemp/Topling-426019-JvfylN 2024/11/23-05:49:02.171331 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247294.sst): first pass time = 0.90's, 149.748'MB/sec 2024/11/23-05:49:03.522212 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247294.sst): old prealloc_size = 147639500, real_size = 33395280 2024/11/23-05:49:03.522810 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247294.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926792 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.82 avg-zval = 33.87 2024/11/23-05:49:03.523598 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247294.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:49:04.472868 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247294.sst): second pass time = 1.35's, 80.225'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.440'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3095.255'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.281'MB/sec write SST data time = 0.13's, 296.858'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33341.089'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.225'MB/sec zip pipeline throughput = 80.225'MB/sec entries = 926792 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.82 avg-zval = 33.87 usrkeys = 926792 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.82 avg-zval = 33.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926792 bytes = 3243792 } seq expand size = 6487544 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4432 value = 3.4493 dict = 1.00 all = 3.5797 } Zip/UnZip{ index = 0.0958 value = 0.2899 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 1.365409 GB avg = 0.123 KB (by entry num) total key len = 0.311731 GB avg = 0.028 KB total ukey len = 0.222665 GB avg = 0.020 KB total ukey num = 0.011133247 Billion total entry num = 0.011133247 Billion write speed all = 0.022242829 MB/sec (with seq num) write speed all = 0.021061604 MB/sec (without seq num) 2024/11/23-05:49:04.505572 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247294: 926792 keys, 37493718 bytes, temperature: kWarm 2024/11/23-05:49:04.505637 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312144505606, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247294, "file_size": 37493718, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3819299798, "table_properties": {"data_size": 31388480, "index_size": 1774912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950176, "raw_average_key_size": 28, "raw_value_size": 108267568, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926792, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312141, "oldest_key_time": 0, "file_creation_time": 1732312141, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247294, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:49:04.541014 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247295.sst) /dev/shm/ToplingTemp/Topling-426019-R80RhK 2024/11/23-05:49:05.466941 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247295.sst): first pass time = 0.93's, 144.961'MB/sec 2024/11/23-05:49:06.791422 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247295.sst): old prealloc_size = 147639500, real_size = 31846224 2024/11/23-05:49:06.792005 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247295.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928174 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0298 GB avg-val = 116.60 avg-zval = 32.16 2024/11/23-05:49:06.792766 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247295.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:49:07.678713 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247295.sst): second pass time = 1.32's, 81.753'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.385'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3144.367'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 283.296'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34584.553'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.753'MB/sec zip pipeline throughput = 81.753'MB/sec entries = 928174 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.60 avg-zval = 32.16 usrkeys = 928174 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.60 avg-zval = 32.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928174 bytes = 3132608 } seq expand size = 6497218 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5130 value = 3.6260 dict = 1.00 all = 3.7457 } Zip/UnZip{ index = 0.0951 value = 0.2758 dict = 1.00 all = 0.2670 } ---------------------------- total value len = 1.480135 GB avg = 0.123 KB (by entry num) total key len = 0.337720 GB avg = 0.028 KB total ukey len = 0.241228 GB avg = 0.020 KB total ukey num = 0.012061421 Billion total entry num = 0.012061421 Billion write speed all = 0.024108016 MB/sec (with seq num) write speed all = 0.022828368 MB/sec (without seq num) 2024/11/23-05:49:07.711055 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247295: 928174 keys, 35832282 bytes, temperature: kWarm 2024/11/23-05:49:07.711117 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312147711085, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247295, "file_size": 35832282, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3720421888, "largest_seqno": 3819299457, "table_properties": {"data_size": 29848256, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25988872, "raw_average_key_size": 28, "raw_value_size": 108228892, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928174, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312144, "oldest_key_time": 0, "file_creation_time": 1732312144, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247295, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:49:07.748445 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247296.sst) /dev/shm/ToplingTemp/Topling-426019-QCRezL 2024/11/23-05:49:08.594065 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247296.sst): first pass time = 0.85's, 158.728'MB/sec 2024/11/23-05:49:09.910799 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247296.sst): old prealloc_size = 147639500, real_size = 32995616 2024/11/23-05:49:09.911341 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247296.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928488 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.56 avg-zval = 33.36 2024/11/23-05:49:09.913867 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247296.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:49:10.847511 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247296.sst): second pass time = 1.32's, 82.240'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.605'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3438.833'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.288'MB/sec write SST data time = 0.13's, 284.210'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37994.932'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.240'MB/sec zip pipeline throughput = 82.240'MB/sec entries = 928488 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.56 avg-zval = 33.36 usrkeys = 928488 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.56 avg-zval = 33.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928488 bytes = 3249728 } seq expand size = 6499416 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.3778 value = 3.4939 dict = 1.00 all = 3.6179 } Zip/UnZip{ index = 0.0964 value = 0.2862 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 1.594855 GB avg = 0.123 KB (by entry num) total key len = 0.363717 GB avg = 0.028 KB total ukey len = 0.259798 GB avg = 0.020 KB total ukey num = 0.012989909 Billion total entry num = 0.012989909 Billion write speed all = 0.025973087 MB/sec (with seq num) write speed all = 0.024594989 MB/sec (without seq num) 2024/11/23-05:49:10.880985 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247296: 928488 keys, 37098612 bytes, temperature: kWarm 2024/11/23-05:49:10.881045 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312150881013, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247296, "file_size": 37098612, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3819299204, "table_properties": {"data_size": 30973904, "index_size": 1789376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997664, "raw_average_key_size": 28, "raw_value_size": 108220116, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928488, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312147, "oldest_key_time": 0, "file_creation_time": 1732312147, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247296, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:49:10.916843 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247297.sst) /dev/shm/ToplingTemp/Topling-426019-53DgjK 2024/11/23-05:49:11.910038 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247297.sst): first pass time = 0.99's, 135.142'MB/sec 2024/11/23-05:49:13.267058 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247297.sst): old prealloc_size = 147639500, real_size = 32558464 2024/11/23-05:49:13.267595 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247297.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934336 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.65 avg-zval = 32.68 2024/11/23-05:49:13.270126 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247297.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:49:14.229011 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247297.sst): second pass time = 1.36's, 79.703'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.131'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3492.426'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.281'MB/sec write SST data time = 0.13's, 283.072'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31063.029'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.703'MB/sec zip pipeline throughput = 79.703'MB/sec entries = 934336 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.65 avg-zval = 32.68 usrkeys = 934336 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.65 avg-zval = 32.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934336 bytes = 3270192 } seq expand size = 6540352 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4443 value = 3.5387 dict = 1.00 all = 3.6592 } Zip/UnZip{ index = 0.0957 value = 0.2826 dict = 1.00 all = 0.2733 } ---------------------------- total value len = 1.709451 GB avg = 0.123 KB (by entry num) total key len = 0.389879 GB avg = 0.028 KB total ukey len = 0.278485 GB avg = 0.020 KB total ukey num = 0.013924245 Billion total entry num = 0.013924245 Billion write speed all = 0.027838465 MB/sec (with seq num) write speed all = 0.026361310 MB/sec (without seq num) 2024/11/23-05:49:14.265672 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247297: 934336 keys, 36679362 bytes, temperature: kWarm 2024/11/23-05:49:14.265736 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312154265704, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247297, "file_size": 36679362, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3819299214, "table_properties": {"data_size": 30535536, "index_size": 1789184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26161408, "raw_average_key_size": 28, "raw_value_size": 108056356, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934336, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312150, "oldest_key_time": 0, "file_creation_time": 1732312150, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247297, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:49:14.301132 140024777004608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247298.sst) /dev/shm/ToplingTemp/Topling-426019-liioXK 2024/11/23-05:49:14.398539 140024777004608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247298.sst): first pass time = 0.10's, 241.076'MB/sec 2024/11/23-05:49:14.733063 140024777004608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247298.sst): old prealloc_size = 147639500, real_size = 6762640 2024/11/23-05:49:14.733211 140024777004608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247298.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 163646 prefix = 4 raw-key = 0.0026 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0189 GB zip-val = 0.0064 GB avg-val = 115.43 avg-zval = 39.16 2024/11/23-05:49:14.733390 140024777004608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247298.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.023 GB 2024/11/23-05:49:14.885006 140024777004608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00/247298.sst): second pass time = 0.33's, 56.574'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 49.462'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2548.442'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.318'MB/sec write SST data time = 0.02's, 325.825'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.198'MB, throughput = 4855.262'MB/sec zip my value time = 0.33's, unzip length = 0.019'GB zip my value throughput = 56.574'MB/sec zip pipeline throughput = 163.417'MB/sec entries = 163646 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.43 avg-zval = 39.16 usrkeys = 163646 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.43 avg-zval = 39.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 163646 bytes = 593232 } seq expand size = 1145522 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0033 GB value = 0.0189 GB dict = 0.20 MB all = 0.0235 GB } __ZipSize{ index = 0.0003 GB value = 0.0064 GB dict = 0.20 MB all = 0.0075 GB } UnZip/Zip{ index = 10.4665 value = 2.9475 dict = 1.00 all = 3.1235 } Zip/UnZip{ index = 0.0955 value = 0.3393 dict = 1.00 all = 0.3202 } ---------------------------- total value len = 1.729487 GB avg = 0.123 KB (by entry num) total key len = 0.394461 GB avg = 0.028 KB total ukey len = 0.281758 GB avg = 0.020 KB total ukey num = 0.014087891 Billion total entry num = 0.014087891 Billion write speed all = 0.028164666 MB/sec (with seq num) write speed all = 0.026670163 MB/sec (without seq num) 2024/11/23-05:49:14.898771 140024777004608 [compaction_job.cc:1948] [default] [JOB 20478] Generated table #247298: 163646 keys, 7514591 bytes, temperature: kWarm 2024/11/23-05:49:14.898844 140024777004608 EVENT_LOG_v1 {"time_micros": 1732312154898804, "cf_name": "default", "job": 20478, "event": "table_file_creation", "file_number": 247298, "file_size": 7514591, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3817784380, "table_properties": {"data_size": 6408832, "index_size": 312704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4582088, "raw_average_key_size": 28, "raw_value_size": 18890000, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 163646, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732312154, "oldest_key_time": 0, "file_creation_time": 1732312154, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247298, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:49:14.909400 140024777004608 [compaction_job.cc:703] job-20478: subcompact[0], size: 0.564920 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20478/att-00 [247283,247284,247285,247286,247287,247288,247289,247290,247291,247292,247293,247294,247295,247296,247297,247298] 2024/11/23-05:49:15.031360 140024777004608 (Original Log Time 2024/11/23-05:48:19.796354) EVENT_LOG_v1 {"time_micros": 1732312099796314, "job": 20478, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [247274], "files_L3": [247219, 247220, 247221, 247222, 247186, 247187, 247188, 247189, 247190, 247191, 247192, 247193, 247194, 247195, 247196, 247197], "score": 0.99792, "input_data_size": 588620529, "oldest_snapshot_seqno": 3819332506} 2024/11/23-05:49:15.031363 140024777004608 (Original Log Time 2024/11/23-05:49:15.030509) [compaction_job.cc:2022] [default] [JOB 20478] Compacted 1@2 + 16@3 files to L3 => 564919707 bytes 2024/11/23-05:49:15.031365 140024777004608 (Original Log Time 2024/11/23-05:49:15.031294) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.7 rd, 10.3 wr, level 3, files in(1, 16) out(16 +0 blob) MB in(27.2, 534.2 +0.0 blob) out(538.7 +0.0 blob), read-write-amplify(40.5) write-amplify(19.8) OK, records in: 14176611, records dropped: 88720 output_compression: Snappy 2024/11/23-05:49:15.031371 140024777004608 (Original Log Time 2024/11/23-05:49:15.031339) EVENT_LOG_v1 {"time_micros": 1732312155031318, "cf": "default", "job": 20478, "event": "compaction_finished", "compaction_time_micros": 55113053, "compaction_time_cpu_micros": 24289127, "output_level": 3, "num_output_files": 16, "total_output_size": 564919707, "num_input_records": 14176611, "num_output_records": 14087891, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 396374312, "file_range_sync_nanos": 0, "file_fsync_nanos": 15421, "file_prepare_write_nanos": 38424, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/23-05:49:15.155959 140024777004608 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732312099,"executesMs":55348,"compactionJobId":20478,"attempt":0,"compactionInputRawBytes":2049337258,"compactionInputZipBytes":588620529,"compactionOutputRawBytes":0,"compactionOutputZipBytes":564919707,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-31d9429ea915a501a113317725f31823-857ce9b8ad767e0d-00"}}, response = 2024/11/23-05:49:15.156030 140024777004608 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20478/att-00: olev 3, work 55.348 s, result 6.290 ms, install 0.879 ms, input{raw 1.909 GiB zip 561.352 MiB}