2024/11/23-09:02:55.485221 139957592835648 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00: fork to child time = 0.007043 sec 2024/11/23-09:02:55.485665 139957592835648 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00 2024/11/23-09:02:55.492137 139957592835648 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00 2024/11/23-09:02:55.493029 139957592835648 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-09:02:55.493252 139957592835648 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-09:02:55.551715 139957592835648 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-23188/att-00/MANIFEST-206347 2024/11/23-09:02:55.583757 139957592835648 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-23188/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 253460, last_sequence is 3897407449, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 253253 2024/11/23-09:02:55.583763 139957592835648 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-09:02:55.583872 139957592835648 [version_set.cc:5721] Creating manifest 253460 2024/11/23-09:02:55.620106 139957592835648 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-09:02:55.643123 139957592835648 [compaction_job.cc:2437] [default] [JOB 23188] Compacting 1@2 + 17@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-09:02:55.643136 139957592835648 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 17406 Base level 2, inputs: [253402(27MB)], [253448(8841KB) 253368(8683KB) 253332(35MB) 253333(35MB) 253334(34MB) 253335(33MB) 253336(34MB) 253337(36MB) 253338(35MB) 253339(35MB) 253340(35MB) 253341(35MB) 253342(35MB) 253343(33MB) 253344(35MB) 253345(35MB) 253449(33MB)] 2024/11/23-09:02:55.649327 139957592835648 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-09:02:55.649354 139957592835648 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-09:02:55.649372 139957592835648 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-09:02:55.649377 139957592835648 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-09:02:55.649384 139957592835648 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-09:02:55.980052 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253461.sst) /dev/shm/ToplingTemp/Topling-7928-kQoZHH 2024/11/23-09:02:57.087825 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253461.sst): first pass time = 1.11's, 121.165'MB/sec 2024/11/23-09:02:58.432896 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253461.sst): old prealloc_size = 147639500, real_size = 31240288 2024/11/23-09:02:58.433410 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253461.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926314 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0292 GB avg-val = 116.89 avg-zval = 31.57 2024/11/23-09:02:58.434225 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253461.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:02:59.405084 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253461.sst): second pass time = 1.34's, 80.560'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 36.468'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3565.729'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.273'MB/sec write SST data time = 0.12's, 290.214'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35731.777'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.560'MB/sec zip pipeline throughput = 80.560'MB/sec entries = 926314 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.89 avg-zval = 31.57 usrkeys = 926314 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.89 avg-zval = 31.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926314 bytes = 3357904 } seq expand size = 6484198 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5133 value = 3.7024 dict = 1.00 all = 3.7858 } Zip/UnZip{ index = 0.0951 value = 0.2701 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 8.963385 GB avg = 0.168 KB (by entry num) total key len = 1.466758 GB avg = 0.028 KB total ukey len = 1.016404 GB avg = 0.015 KB total ukey num = 0.066476749 Billion total entry num = 0.053284478 Billion write speed all = 22.325249559 MB/sec (with seq num) write speed all = 21.412825418 MB/sec (without seq num) 2024/11/23-09:02:59.454125 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253461: 926314 keys, 35452818 bytes, temperature: kWarm 2024/11/23-09:02:59.454218 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323779454160, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253461, "file_size": 35452818, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3897078280, "table_properties": {"data_size": 29246352, "index_size": 1762176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25936792, "raw_average_key_size": 28, "raw_value_size": 108281016, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926314, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323775, "oldest_key_time": 0, "file_creation_time": 1732323775, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253461, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:02:59.486014 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253462.sst) /dev/shm/ToplingTemp/Topling-7928-1xuphE 2024/11/23-09:03:00.669483 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253462.sst): first pass time = 1.18's, 113.414'MB/sec 2024/11/23-09:03:01.972094 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253462.sst): old prealloc_size = 147639500, real_size = 33946112 2024/11/23-09:03:01.972586 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253462.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929123 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0319 GB avg-val = 116.46 avg-zval = 34.38 2024/11/23-09:03:01.973436 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253462.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:02.815512 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253462.sst): second pass time = 1.30's, 83.124'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.254'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3740.320'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 315.147'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33840.595'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.124'MB/sec zip pipeline throughput = 83.124'MB/sec entries = 929123 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.46 avg-zval = 34.38 usrkeys = 929123 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.46 avg-zval = 34.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929123 bytes = 3368080 } seq expand size = 6503861 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5135 value = 3.3870 dict = 1.00 all = 3.5166 } Zip/UnZip{ index = 0.0951 value = 0.2952 dict = 1.00 all = 0.2844 } ---------------------------- total value len = 9.078091 GB avg = 0.167 KB (by entry num) total key len = 1.492773 GB avg = 0.028 KB total ukey len = 1.034987 GB avg = 0.015 KB total ukey num = 0.067405872 Billion total entry num = 0.054213601 Billion write speed all = 22.462484645 MB/sec (with seq num) write speed all = 21.540878145 MB/sec (without seq num) 2024/11/23-09:03:02.864355 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253462: 929123 keys, 38166632 bytes, temperature: kWarm 2024/11/23-09:03:02.864422 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323782864387, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253462, "file_size": 38166632, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3897078161, "table_properties": {"data_size": 31946160, "index_size": 1767488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26015444, "raw_average_key_size": 28, "raw_value_size": 108202322, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929123, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323779, "oldest_key_time": 0, "file_creation_time": 1732323779, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253462, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:02.898684 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253463.sst) /dev/shm/ToplingTemp/Topling-7928-m6MpwG 2024/11/23-09:03:03.917013 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253463.sst): first pass time = 1.02's, 131.807'MB/sec 2024/11/23-09:03:05.190478 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253463.sst): old prealloc_size = 147639500, real_size = 32773696 2024/11/23-09:03:05.191048 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253463.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929705 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.37 avg-zval = 33.09 2024/11/23-09:03:05.191855 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253463.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:06.035371 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253463.sst): second pass time = 1.27's, 85.006'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.652'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3222.011'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.12's, 306.526'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35706.081'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.006'MB/sec zip pipeline throughput = 85.006'MB/sec entries = 929705 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.37 avg-zval = 33.09 usrkeys = 929705 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.37 avg-zval = 33.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929705 bytes = 3253984 } seq expand size = 6507935 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4433 value = 3.5170 dict = 1.00 all = 3.6393 } Zip/UnZip{ index = 0.0958 value = 0.2843 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 9.192785 GB avg = 0.167 KB (by entry num) total key len = 1.518805 GB avg = 0.028 KB total ukey len = 1.053581 GB avg = 0.015 KB total ukey num = 0.068335577 Billion total entry num = 0.055143306 Billion write speed all = 22.606842233 MB/sec (with seq num) write speed all = 21.675801372 MB/sec (without seq num) 2024/11/23-09:03:06.081029 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253463: 929705 keys, 36880420 bytes, temperature: kWarm 2024/11/23-09:03:06.081096 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323786081061, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253463, "file_size": 36880420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 3897077659, "table_properties": {"data_size": 30760624, "index_size": 1780480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26031740, "raw_average_key_size": 28, "raw_value_size": 108186006, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929705, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323782, "oldest_key_time": 0, "file_creation_time": 1732323782, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253463, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:06.117351 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253464.sst) /dev/shm/ToplingTemp/Topling-7928-hLlSlH 2024/11/23-09:03:07.118992 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253464.sst): first pass time = 1.00's, 134.003'MB/sec 2024/11/23-09:03:08.385523 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253464.sst): old prealloc_size = 147639500, real_size = 32985888 2024/11/23-09:03:08.386118 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253464.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929453 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.41 avg-zval = 33.34 2024/11/23-09:03:08.387015 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253464.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:09.230394 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253464.sst): second pass time = 1.27's, 85.491'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.766'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3082.963'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.12's, 305.772'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34895.397'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.491'MB/sec zip pipeline throughput = 85.491'MB/sec entries = 929453 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.41 avg-zval = 33.34 usrkeys = 929453 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.41 avg-zval = 33.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929453 bytes = 3136912 } seq expand size = 6506171 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5153 value = 3.4917 dict = 1.00 all = 3.6299 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 9.307484 GB avg = 0.166 KB (by entry num) total key len = 1.544830 GB avg = 0.028 KB total ukey len = 1.072170 GB avg = 0.015 KB total ukey num = 0.069265030 Billion total entry num = 0.056072759 Billion write speed all = 22.750431930 MB/sec (with seq num) write speed all = 21.810039328 MB/sec (without seq num) 2024/11/23-09:03:09.275788 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253464: 929453 keys, 36975796 bytes, temperature: kWarm 2024/11/23-09:03:09.275855 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323789275820, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253464, "file_size": 36975796, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 3897078063, "table_properties": {"data_size": 30985552, "index_size": 1767808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26024684, "raw_average_key_size": 28, "raw_value_size": 108193058, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929453, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323786, "oldest_key_time": 0, "file_creation_time": 1732323786, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253464, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:09.309238 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253465.sst) /dev/shm/ToplingTemp/Topling-7928-G88LZG 2024/11/23-09:03:10.269391 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253465.sst): first pass time = 0.96's, 139.794'MB/sec 2024/11/23-09:03:11.606766 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253465.sst): old prealloc_size = 147639500, real_size = 34083120 2024/11/23-09:03:11.607314 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253465.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933116 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0321 GB avg-val = 115.84 avg-zval = 34.37 2024/11/23-09:03:11.607810 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253465.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:12.386573 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253465.sst): second pass time = 1.34's, 80.870'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 47.179'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3366.096'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.355'MB/sec write SST data time = 0.12's, 301.697'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31992.107'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.870'MB/sec zip pipeline throughput = 80.870'MB/sec entries = 933116 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.84 avg-zval = 34.37 usrkeys = 933116 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.63 avg-zval = 34.37 TagRS{ kinds = RS_Key0_TagN bytes = 145880 } TagArray{ size = 505674 bytes = 1706672 } seq expand size = 3539718 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5145 value = 3.3700 dict = 1.00 all = 3.6482 } Zip/UnZip{ index = 0.0951 value = 0.2967 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 9.419115 GB avg = 0.165 KB (by entry num) total key len = 1.570957 GB avg = 0.028 KB total ukey len = 1.090832 GB avg = 0.016 KB total ukey num = 0.070198146 Billion total entry num = 0.057005875 Billion write speed all = 22.887784941 MB/sec (with seq num) write speed all = 21.938027191 MB/sec (without seq num) 2024/11/23-09:03:12.431263 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253465: 933116 keys, 36790085 bytes, temperature: kWarm 2024/11/23-09:03:12.431328 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323792431293, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253465, "file_size": 36790085, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3897076792, "table_properties": {"data_size": 32074720, "index_size": 1774912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26127248, "raw_average_key_size": 28, "raw_value_size": 108090512, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933116, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323789, "oldest_key_time": 0, "file_creation_time": 1732323789, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253465, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:12.464359 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253466.sst) /dev/shm/ToplingTemp/Topling-7928-wbHrBH 2024/11/23-09:03:13.522171 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253466.sst): first pass time = 1.06's, 126.887'MB/sec 2024/11/23-09:03:14.840429 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253466.sst): old prealloc_size = 147639500, real_size = 35498832 2024/11/23-09:03:14.841049 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253466.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080669 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1040 GB zip-val = 0.0332 GB avg-val = 96.20 avg-zval = 30.68 2024/11/23-09:03:14.841279 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253466.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:15.530088 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253466.sst): second pass time = 1.32's, 78.909'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 53.420'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3455.421'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.490'MB/sec write SST data time = 0.14's, 267.428'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.028'MB, throughput = 34470.164'MB/sec zip my value time = 1.32's, unzip length = 0.104'GB zip my value throughput = 78.909'MB/sec zip pipeline throughput = 78.909'MB/sec entries = 1080669 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.20 avg-zval = 30.68 usrkeys = 1080669 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.97 avg-zval = 30.68 TagRS{ kinds = RS_Key0_TagN bytes = 168960 } TagArray{ size = 119708 bytes = 404032 } seq expand size = 837956 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0332 GB dict = 1.03 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4470 value = 3.1351 dict = 1.00 all = 3.6432 } Zip/UnZip{ index = 0.0957 value = 0.3190 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 9.523912 GB avg = 0.164 KB (by entry num) total key len = 1.601216 GB avg = 0.028 KB total ukey len = 1.112446 GB avg = 0.016 KB total ukey num = 0.071278815 Billion total entry num = 0.058086544 Billion write speed all = 23.018357296 MB/sec (with seq num) write speed all = 22.056889236 MB/sec (without seq num) 2024/11/23-09:03:15.576596 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253466: 1080669 keys, 36840484 bytes, temperature: kWarm 2024/11/23-09:03:15.576663 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323795576628, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253466, "file_size": 36840484, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3404355789, "table_properties": {"data_size": 33159616, "index_size": 2068864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30258732, "raw_average_key_size": 28, "raw_value_size": 103959071, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080669, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323792, "oldest_key_time": 0, "file_creation_time": 1732323792, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253466, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:15.610708 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253467.sst) /dev/shm/ToplingTemp/Topling-7928-eEi5VD 2024/11/23-09:03:16.634579 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253467.sst): first pass time = 1.02's, 131.095'MB/sec 2024/11/23-09:03:17.855764 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253467.sst): old prealloc_size = 147639500, real_size = 30980832 2024/11/23-09:03:17.856310 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253467.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925150 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 117.08 avg-zval = 31.32 2024/11/23-09:03:17.857094 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253467.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:18.666779 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253467.sst): second pass time = 1.22's, 88.753'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.389'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3378.223'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.12's, 285.927'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37823.169'MB/sec zip my value time = 1.22's, unzip length = 0.108'GB zip my value throughput = 88.753'MB/sec zip pipeline throughput = 88.753'MB/sec entries = 925150 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.08 avg-zval = 31.32 usrkeys = 925150 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.08 avg-zval = 31.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925150 bytes = 3353680 } seq expand size = 6476050 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4447 value = 3.7378 dict = 1.00 all = 3.8143 } Zip/UnZip{ index = 0.0957 value = 0.2675 dict = 1.00 all = 0.2622 } ---------------------------- total value len = 9.638701 GB avg = 0.163 KB (by entry num) total key len = 1.627120 GB avg = 0.028 KB total ukey len = 1.130949 GB avg = 0.016 KB total ukey num = 0.072203965 Billion total entry num = 0.059011694 Billion write speed all = 23.159157655 MB/sec (with seq num) write speed all = 22.188674566 MB/sec (without seq num) 2024/11/23-09:03:18.708785 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253467: 925150 keys, 35188240 bytes, temperature: kWarm 2024/11/23-09:03:18.708851 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323798708816, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253467, "file_size": 35188240, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3404355790, "largest_seqno": 3897076851, "table_properties": {"data_size": 28977872, "index_size": 1771520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904200, "raw_average_key_size": 28, "raw_value_size": 108313652, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925150, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323795, "oldest_key_time": 0, "file_creation_time": 1732323795, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253467, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:18.752383 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253468.sst) /dev/shm/ToplingTemp/Topling-7928-J8Zp9E 2024/11/23-09:03:19.685874 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253468.sst): first pass time = 0.93's, 143.787'MB/sec 2024/11/23-09:03:20.962192 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253468.sst): old prealloc_size = 147639500, real_size = 32045744 2024/11/23-09:03:20.962690 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253468.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929897 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0300 GB avg-val = 116.34 avg-zval = 32.31 2024/11/23-09:03:20.964139 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253468.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:21.799088 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253468.sst): second pass time = 1.28's, 84.817'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.338'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3717.325'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.12's, 294.689'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35657.522'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.817'MB/sec zip pipeline throughput = 84.817'MB/sec entries = 929897 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.34 avg-zval = 32.31 usrkeys = 929897 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.34 avg-zval = 32.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929897 bytes = 3370896 } seq expand size = 6509279 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5120 value = 3.6008 dict = 1.00 all = 3.7006 } Zip/UnZip{ index = 0.0951 value = 0.2777 dict = 1.00 all = 0.2702 } ---------------------------- total value len = 9.753391 GB avg = 0.163 KB (by entry num) total key len = 1.653157 GB avg = 0.028 KB total ukey len = 1.149547 GB avg = 0.016 KB total ukey num = 0.073133862 Billion total entry num = 0.059941591 Billion write speed all = 23.298429251 MB/sec (with seq num) write speed all = 22.318960351 MB/sec (without seq num) 2024/11/23-09:03:21.842943 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253468: 929897 keys, 36268886 bytes, temperature: kWarm 2024/11/23-09:03:21.843027 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323801842990, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253468, "file_size": 36268886, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3897076865, "table_properties": {"data_size": 30043872, "index_size": 1769216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26037116, "raw_average_key_size": 28, "raw_value_size": 108180690, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929897, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323798, "oldest_key_time": 0, "file_creation_time": 1732323798, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253468, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:21.876223 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253469.sst) /dev/shm/ToplingTemp/Topling-7928-tu5XWF 2024/11/23-09:03:22.900942 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253469.sst): first pass time = 1.02's, 130.985'MB/sec 2024/11/23-09:03:24.159620 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253469.sst): old prealloc_size = 147639500, real_size = 31563168 2024/11/23-09:03:24.160182 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253469.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932175 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0295 GB avg-val = 115.98 avg-zval = 31.69 2024/11/23-09:03:24.161634 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253469.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:24.991681 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253469.sst): second pass time = 1.26's, 85.966'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.738'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3321.346'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.12's, 290.651'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36151.204'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.966'MB/sec zip pipeline throughput = 85.966'MB/sec entries = 932175 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.98 avg-zval = 31.69 usrkeys = 932175 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.98 avg-zval = 31.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932175 bytes = 3146112 } seq expand size = 6525225 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4429 value = 3.6594 dict = 1.00 all = 3.7744 } Zip/UnZip{ index = 0.0958 value = 0.2733 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 9.868033 GB avg = 0.162 KB (by entry num) total key len = 1.679258 GB avg = 0.028 KB total ukey len = 1.168190 GB avg = 0.016 KB total ukey num = 0.074066037 Billion total entry num = 0.060873766 Billion write speed all = 23.433095641 MB/sec (with seq num) write speed all = 22.444839093 MB/sec (without seq num) 2024/11/23-09:03:25.034781 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253469: 932175 keys, 35559876 bytes, temperature: kWarm 2024/11/23-09:03:25.034847 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323805034813, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253469, "file_size": 35559876, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 3897077639, "table_properties": {"data_size": 29544656, "index_size": 1785280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26100900, "raw_average_key_size": 28, "raw_value_size": 108116862, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932175, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323801, "oldest_key_time": 0, "file_creation_time": 1732323801, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253469, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:25.076320 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253470.sst) /dev/shm/ToplingTemp/Topling-7928-Rt7LhE 2024/11/23-09:03:26.070103 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253470.sst): first pass time = 0.99's, 135.063'MB/sec 2024/11/23-09:03:27.327271 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253470.sst): old prealloc_size = 147639500, real_size = 31206288 2024/11/23-09:03:27.327795 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253470.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927767 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0292 GB avg-val = 116.67 avg-zval = 31.48 2024/11/23-09:03:27.329260 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253470.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:28.144930 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253470.sst): second pass time = 1.26's, 86.151'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.466'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3530.665'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.12's, 290.934'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34309.124'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 86.151'MB/sec zip pipeline throughput = 86.151'MB/sec entries = 927767 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.67 avg-zval = 31.48 usrkeys = 927767 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.67 avg-zval = 31.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927767 bytes = 3479136 } seq expand size = 6494369 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5118 value = 3.7057 dict = 1.00 all = 3.7766 } Zip/UnZip{ index = 0.0951 value = 0.2699 dict = 1.00 all = 0.2648 } ---------------------------- total value len = 9.982768 GB avg = 0.162 KB (by entry num) total key len = 1.705235 GB avg = 0.028 KB total ukey len = 1.186745 GB avg = 0.016 KB total ukey num = 0.074993804 Billion total entry num = 0.061801533 Billion write speed all = 23.567836069 MB/sec (with seq num) write speed all = 22.570897009 MB/sec (without seq num) 2024/11/23-09:03:28.187258 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253470: 927767 keys, 35539640 bytes, temperature: kWarm 2024/11/23-09:03:28.187324 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323808187288, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253470, "file_size": 35539640, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3897077746, "table_properties": {"data_size": 29208960, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25977476, "raw_average_key_size": 28, "raw_value_size": 108240306, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927767, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323805, "oldest_key_time": 0, "file_creation_time": 1732323805, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253470, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:28.223578 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253471.sst) /dev/shm/ToplingTemp/Topling-7928-T8ZlGE 2024/11/23-09:03:29.160705 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253471.sst): first pass time = 0.94's, 143.229'MB/sec 2024/11/23-09:03:30.432612 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253471.sst): old prealloc_size = 147639500, real_size = 31343200 2024/11/23-09:03:30.433191 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253471.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931372 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0293 GB avg-val = 116.11 avg-zval = 31.50 2024/11/23-09:03:30.434646 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253471.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:31.268741 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253471.sst): second pass time = 1.27's, 85.075'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.404'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3172.326'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.12's, 289.169'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36393.586'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.075'MB/sec zip pipeline throughput = 85.075'MB/sec entries = 931372 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.11 avg-zval = 31.50 usrkeys = 931372 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.11 avg-zval = 31.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931372 bytes = 3259824 } seq expand size = 6519604 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5134 value = 3.6859 dict = 1.00 all = 3.7857 } Zip/UnZip{ index = 0.0951 value = 0.2713 dict = 1.00 all = 0.2642 } ---------------------------- total value len = 10.097427 GB avg = 0.161 KB (by entry num) total key len = 1.731314 GB avg = 0.028 KB total ukey len = 1.205373 GB avg = 0.016 KB total ukey num = 0.075925176 Billion total entry num = 0.062732905 Billion write speed all = 23.702322288 MB/sec (with seq num) write speed all = 22.696693336 MB/sec (without seq num) 2024/11/23-09:03:31.311255 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253471: 931372 keys, 35454112 bytes, temperature: kWarm 2024/11/23-09:03:31.311324 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323811311289, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253471, "file_size": 35454112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3897076910, "table_properties": {"data_size": 29338384, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26078416, "raw_average_key_size": 28, "raw_value_size": 108139360, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931372, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323808, "oldest_key_time": 0, "file_creation_time": 1732323808, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253471, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:31.345565 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253472.sst) /dev/shm/ToplingTemp/Topling-7928-NP54wF 2024/11/23-09:03:32.333654 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253472.sst): first pass time = 0.99's, 135.841'MB/sec 2024/11/23-09:03:33.569896 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253472.sst): old prealloc_size = 147639500, real_size = 31844528 2024/11/23-09:03:33.570411 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253472.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920164 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0299 GB avg-val = 117.86 avg-zval = 32.45 2024/11/23-09:03:33.571893 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253472.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:34.412351 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253472.sst): second pass time = 1.24's, 87.783'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.781'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3556.863'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 278.658'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 36291.978'MB/sec zip my value time = 1.24's, unzip length = 0.108'GB zip my value throughput = 87.783'MB/sec zip pipeline throughput = 87.783'MB/sec entries = 920164 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.86 avg-zval = 32.45 usrkeys = 920164 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.86 avg-zval = 32.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920164 bytes = 3335616 } seq expand size = 6441148 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5115 value = 3.6316 dict = 1.00 all = 3.7241 } Zip/UnZip{ index = 0.0951 value = 0.2754 dict = 1.00 all = 0.2685 } ---------------------------- total value len = 10.212321 GB avg = 0.160 KB (by entry num) total key len = 1.757078 GB avg = 0.028 KB total ukey len = 1.223776 GB avg = 0.016 KB total ukey num = 0.076845340 Billion total entry num = 0.063653069 Billion write speed all = 23.834039510 MB/sec (with seq num) write speed all = 22.820047302 MB/sec (without seq num) 2024/11/23-09:03:34.455550 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253472: 920164 keys, 36040352 bytes, temperature: kWarm 2024/11/23-09:03:34.455618 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323814455583, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253472, "file_size": 36040352, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3897070799, "table_properties": {"data_size": 29863520, "index_size": 1750784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25764592, "raw_average_key_size": 28, "raw_value_size": 108453176, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920164, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323811, "oldest_key_time": 0, "file_creation_time": 1732323811, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253472, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:34.488066 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253473.sst) /dev/shm/ToplingTemp/Topling-7928-TfVjNH 2024/11/23-09:03:35.513630 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253473.sst): first pass time = 1.03's, 130.877'MB/sec 2024/11/23-09:03:36.801326 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253473.sst): old prealloc_size = 147639500, real_size = 33196416 2024/11/23-09:03:36.801856 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253473.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 937456 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0312 GB avg-val = 115.17 avg-zval = 33.26 2024/11/23-09:03:36.803326 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253473.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:37.649448 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253473.sst): second pass time = 1.29's, 83.913'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.149'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3517.273'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 298.178'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.076'MB, throughput = 35873.292'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.913'MB/sec zip pipeline throughput = 83.913'MB/sec entries = 937456 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.17 avg-zval = 33.26 usrkeys = 937456 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.17 avg-zval = 33.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 937456 bytes = 3398288 } seq expand size = 6562192 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5137 value = 3.4629 dict = 1.00 all = 3.5846 } Zip/UnZip{ index = 0.0951 value = 0.2888 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 10.326853 GB avg = 0.160 KB (by entry num) total key len = 1.783327 GB avg = 0.028 KB total ukey len = 1.242525 GB avg = 0.016 KB total ukey num = 0.077782796 Billion total entry num = 0.064590525 Billion write speed all = 23.959925004 MB/sec (with seq num) write speed all = 22.937588975 MB/sec (without seq num) 2024/11/23-09:03:37.695853 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253473: 937456 keys, 37442638 bytes, temperature: kWarm 2024/11/23-09:03:37.695922 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323817695883, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253473, "file_size": 37442638, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3897078005, "table_properties": {"data_size": 31178544, "index_size": 1783296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26248768, "raw_average_key_size": 28, "raw_value_size": 107969028, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 937456, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323814, "oldest_key_time": 0, "file_creation_time": 1732323814, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253473, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:37.728739 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253474.sst) /dev/shm/ToplingTemp/Topling-7928-RWxImH 2024/11/23-09:03:38.576132 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253474.sst): first pass time = 0.85's, 158.399'MB/sec 2024/11/23-09:03:39.858919 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253474.sst): old prealloc_size = 147639500, real_size = 32671792 2024/11/23-09:03:39.859433 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253474.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931869 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.03 avg-zval = 32.91 2024/11/23-09:03:39.860904 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253474.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:40.718040 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253474.sst): second pass time = 1.28's, 84.363'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.954'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3565.115'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.12's, 295.692'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35327.434'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.363'MB/sec zip pipeline throughput = 84.363'MB/sec entries = 931869 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 32.91 usrkeys = 931869 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 32.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931869 bytes = 3378048 } seq expand size = 6523083 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5130 value = 3.5259 dict = 1.00 all = 3.6373 } Zip/UnZip{ index = 0.0951 value = 0.2836 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 10.441501 GB avg = 0.159 KB (by entry num) total key len = 1.809420 GB avg = 0.028 KB total ukey len = 1.261163 GB avg = 0.016 KB total ukey num = 0.078714665 Billion total entry num = 0.065522394 Billion write speed all = 24.092113245 MB/sec (with seq num) write speed all = 23.061285941 MB/sec (without seq num) 2024/11/23-09:03:40.763695 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253474: 931869 keys, 36900576 bytes, temperature: kWarm 2024/11/23-09:03:40.763776 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323820763728, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253474, "file_size": 36900576, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3897077144, "table_properties": {"data_size": 30665824, "index_size": 1772800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26092332, "raw_average_key_size": 28, "raw_value_size": 108125470, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931869, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323817, "oldest_key_time": 0, "file_creation_time": 1732323817, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253474, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:40.799651 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253475.sst) /dev/shm/ToplingTemp/Topling-7928-WprglE 2024/11/23-09:03:41.794282 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253475.sst): first pass time = 0.99's, 134.948'MB/sec 2024/11/23-09:03:43.090678 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253475.sst): old prealloc_size = 147639500, real_size = 32972016 2024/11/23-09:03:43.091221 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253475.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924969 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.11 avg-zval = 33.48 2024/11/23-09:03:43.093194 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253475.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-09:03:43.932174 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253475.sst): second pass time = 1.30's, 83.604'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.185'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3391.001'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 296.429'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 22163.953'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.604'MB/sec zip pipeline throughput = 83.604'MB/sec entries = 924969 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.11 avg-zval = 33.48 usrkeys = 924969 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.11 avg-zval = 33.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924969 bytes = 3353024 } seq expand size = 6474783 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4427 value = 3.4976 dict = 1.00 all = 3.6099 } Zip/UnZip{ index = 0.0958 value = 0.2859 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 10.556295 GB avg = 0.159 KB (by entry num) total key len = 1.835319 GB avg = 0.028 KB total ukey len = 1.279662 GB avg = 0.016 KB total ukey num = 0.079639634 Billion total entry num = 0.066447363 Billion write speed all = 24.215729792 MB/sec (with seq num) write speed all = 23.176916611 MB/sec (without seq num) 2024/11/23-09:03:43.978016 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253475: 924969 keys, 37180270 bytes, temperature: kWarm 2024/11/23-09:03:43.978081 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323823978047, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253475, "file_size": 37180270, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3897077198, "table_properties": {"data_size": 30969056, "index_size": 1771520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899132, "raw_average_key_size": 28, "raw_value_size": 108318634, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924969, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323820, "oldest_key_time": 0, "file_creation_time": 1732323820, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253475, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:44.013562 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253476.sst) /dev/shm/ToplingTemp/Topling-7928-qCVfyG 2024/11/23-09:03:44.332334 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253476.sst): first pass time = 0.32's, 245.730'MB/sec 2024/11/23-09:03:45.077430 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253476.sst): old prealloc_size = 147639500, real_size = 19308016 2024/11/23-09:03:45.077782 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253476.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 547916 prefix = 4 raw-key = 0.0088 GB zip-key = 0.0010 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0630 GB zip-val = 0.0181 GB avg-val = 114.94 avg-zval = 33.09 2024/11/23-09:03:45.078214 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253476.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0014 GB, waited 0.000 sec, Key+Value = 0.078 GB 2024/11/23-09:03:45.563089 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00/253476.sst): second pass time = 0.74's, 84.611'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.49's, 43.982'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3149.423'MB/sec (index lex order gen) rebuild zvType time = 0.41's, 0.334'MB/sec write SST data time = 0.08's, 283.277'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.633'MB, throughput = 21201.072'MB/sec zip my value time = 0.74's, unzip length = 0.063'GB zip my value throughput = 84.611'MB/sec zip pipeline throughput = 84.611'MB/sec entries = 547916 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.94 avg-zval = 33.09 usrkeys = 547916 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.70 avg-zval = 33.09 TagRS{ kinds = RS_Key0_TagN bytes = 85720 } TagArray{ size = 450225 bytes = 1463248 } seq expand size = 3151575 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0110 GB value = 0.0630 GB dict = 0.63 MB all = 0.0783 GB } __ZipSize{ index = 0.0010 GB value = 0.0181 GB dict = 0.63 MB all = 0.0214 GB } UnZip/Zip{ index = 10.5071 value = 3.4742 dict = 1.00 all = 3.6669 } Zip/UnZip{ index = 0.0952 value = 0.2878 dict = 1.00 all = 0.2727 } ---------------------------- total value len = 10.622426 GB avg = 0.159 KB (by entry num) total key len = 1.850660 GB avg = 0.028 KB total ukey len = 1.290620 GB avg = 0.016 KB total ukey num = 0.080187550 Billion total entry num = 0.066995279 Billion write speed all = 24.297504870 MB/sec (with seq num) write speed all = 23.253453311 MB/sec (without seq num) 2024/11/23-09:03:45.595832 139957592835648 [compaction_job.cc:1948] [default] [JOB 23188] Generated table #253476: 547916 keys, 21358841 bytes, temperature: kWarm 2024/11/23-09:03:45.595898 139957592835648 EVENT_LOG_v1 {"time_micros": 1732323825595863, "cf_name": "default", "job": 23188, "event": "table_file_creation", "file_number": 253476, "file_size": 21358841, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3897076983, "table_properties": {"data_size": 18127904, "index_size": 1042944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 15341648, "raw_average_key_size": 28, "raw_value_size": 62979652, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 547916, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323824, "oldest_key_time": 0, "file_creation_time": 1732323824, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253476, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-09:03:45.613701 139957592835648 [compaction_job.cc:703] job-23188: subcompact[0], size: 0.568040 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23188/att-00 [253461,253462,253463,253464,253465,253466,253467,253468,253469,253470,253471,253472,253473,253474,253475,253476] 2024/11/23-09:03:45.698448 139957592835648 (Original Log Time 2024/11/23-09:02:55.643179) EVENT_LOG_v1 {"time_micros": 1732323775643144, "job": 23188, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [253402], "files_L3": [253448, 253368, 253332, 253333, 253334, 253335, 253336, 253337, 253338, 253339, 253340, 253341, 253342, 253343, 253344, 253345, 253449], "score": 0.991332, "input_data_size": 598623325, "oldest_snapshot_seqno": 3897407424} 2024/11/23-09:03:45.698452 139957592835648 (Original Log Time 2024/11/23-09:03:45.697142) [compaction_job.cc:2022] [default] [JOB 23188] Compacted 1@2 + 17@3 files to L3 => 568039666 bytes 2024/11/23-09:03:45.698453 139957592835648 (Original Log Time 2024/11/23-09:03:45.698383) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 12.0 rd, 11.4 wr, level 3, files in(1, 17) out(16 +0 blob) MB in(27.1, 543.8 +0.0 blob) out(541.7 +0.0 blob), read-write-amplify(41.1) write-amplify(20.0) OK, records in: 14718162, records dropped: 81047 output_compression: Snappy 2024/11/23-09:03:45.698459 139957592835648 (Original Log Time 2024/11/23-09:03:45.698426) EVENT_LOG_v1 {"time_micros": 1732323825698407, "cf": "default", "job": 23188, "event": "compaction_finished", "compaction_time_micros": 49970532, "compaction_time_cpu_micros": 21790450, "output_level": 3, "num_output_files": 16, "total_output_size": 568039666, "num_input_records": 14718162, "num_output_records": 14637115, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 377147362, "file_range_sync_nanos": 0, "file_fsync_nanos": 10178, "file_prepare_write_nanos": 40200, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/23-09:03:45.851148 139957592835648 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732323775,"executesMs":50204,"compactionJobId":23188,"attempt":0,"compactionInputRawBytes":2103096709,"compactionInputZipBytes":598623325,"compactionOutputRawBytes":0,"compactionOutputZipBytes":568039666,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-6850bece0c4f95651f5f2817bd9ea6e5-e081b8297181e35f-00"}}, response = 2024/11/23-09:03:45.851240 139957592835648 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-23188/att-00: olev 3, work 50.204 s, result 7.688 ms, install 1.334 ms, input{raw 1.959 GiB zip 570.892 MiB}