2024/11/22-22:35:08.918675 140025137890880 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00: fork to child time = 0.011451 sec 2024/11/22-22:35:08.919612 140025137890880 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00 2024/11/22-22:35:08.927070 140025137890880 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00 2024/11/22-22:35:08.928184 140025137890880 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-22:35:08.928448 140025137890880 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-22:35:08.972365 140025137890880 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12905/att-00/MANIFEST-206347 2024/11/22-22:35:08.995851 140025137890880 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12905/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 231609, last_sequence is 3613614043, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 231463 2024/11/22-22:35:08.995859 140025137890880 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-22:35:08.995953 140025137890880 [version_set.cc:5721] Creating manifest 231609 2024/11/22-22:35:09.034838 140025137890880 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-22:35:09.053566 140025137890880 [compaction_job.cc:2437] [default] [JOB 12905] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-22:35:09.053586 140025137890880 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 9707 Base level 2, inputs: [231598(27MB)], [231533(35MB) 231534(35MB) 231535(35MB) 231536(35MB) 231537(35MB) 231538(35MB) 231539(34MB) 231540(36MB) 231541(36MB) 231542(35MB) 231543(36MB) 231544(35MB) 231545(36MB) 231546(34MB) 231562(35MB)] 2024/11/22-22:35:09.059579 140025137890880 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:35:09.059603 140025137890880 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:35:09.059619 140025137890880 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:35:09.059624 140025137890880 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:35:09.059633 140025137890880 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:35:09.505566 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231610.sst) /dev/shm/ToplingTemp/Topling-317308-iaScRx 2024/11/22-22:35:10.502517 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231610.sst): first pass time = 1.00's, 134.634'MB/sec 2024/11/22-22:35:14.025114 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231610.sst): old prealloc_size = 147639500, real_size = 33049824 2024/11/22-22:35:14.025701 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231610.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924801 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.13 avg-zval = 33.58 2024/11/22-22:35:14.026539 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231610.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:15.381011 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231610.sst): second pass time = 3.51's, 30.873'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.36's, 27.308'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3091.948'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.226'MB/sec write SST data time = 0.33's, 111.252'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31463.869'MB/sec zip my value time = 3.51's, unzip length = 0.108'GB zip my value throughput = 30.873'MB/sec zip pipeline throughput = 125.384'MB/sec entries = 924801 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.13 avg-zval = 33.58 usrkeys = 924801 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.13 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924801 bytes = 3121216 } seq expand size = 6473607 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5148 value = 3.4876 dict = 1.00 all = 3.6249 } Zip/UnZip{ index = 0.0951 value = 0.2867 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 0.114797 GB avg = 0.124 KB (by entry num) total key len = 0.025894 GB avg = 0.028 KB total ukey len = 0.018496 GB avg = 0.020 KB total ukey num = 0.000924801 Billion total entry num = 0.000924801 Billion write speed all = 0.002849602 MB/sec (with seq num) write speed all = 0.002699752 MB/sec (without seq num) 2024/11/22-22:35:15.417163 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231610: 924801 keys, 37026810 bytes, temperature: kWarm 2024/11/22-22:35:15.417280 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286115417203, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231610, "file_size": 37026810, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3513012344, "largest_seqno": 3613582443, "table_properties": {"data_size": 31059408, "index_size": 1759040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25894428, "raw_average_key_size": 28, "raw_value_size": 108323322, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924801, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286109, "oldest_key_time": 0, "file_creation_time": 1732286109, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231610, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:15.493127 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231611.sst) /dev/shm/ToplingTemp/Topling-317308-Waps9w 2024/11/22-22:35:17.145364 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231611.sst): first pass time = 1.65's, 81.236'MB/sec 2024/11/22-22:35:19.496982 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231611.sst): old prealloc_size = 147639500, real_size = 33538176 2024/11/22-22:35:19.497510 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231611.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924574 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.17 avg-zval = 34.12 2024/11/22-22:35:19.498299 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231611.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:20.918263 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231611.sst): second pass time = 2.35's, 46.084'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.42's, 26.477'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3495.268'MB/sec (index lex order gen) rebuild zvType time = 1.11's, 0.208'MB/sec write SST data time = 0.31's, 120.563'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 18387.962'MB/sec zip my value time = 2.35's, unzip length = 0.108'GB zip my value throughput = 46.084'MB/sec zip pipeline throughput = 82.294'MB/sec entries = 924574 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.17 avg-zval = 34.12 usrkeys = 924574 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.17 avg-zval = 34.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924574 bytes = 3236016 } seq expand size = 6472018 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5126 value = 3.4338 dict = 1.00 all = 3.5667 } Zip/UnZip{ index = 0.0951 value = 0.2912 dict = 1.00 all = 0.2804 } ---------------------------- total value len = 0.229599 GB avg = 0.124 KB (by entry num) total key len = 0.051783 GB avg = 0.028 KB total ukey len = 0.036987 GB avg = 0.020 KB total ukey num = 0.001849375 Billion total entry num = 0.001849375 Billion write speed all = 0.005698534 MB/sec (with seq num) write speed all = 0.005398906 MB/sec (without seq num) 2024/11/22-22:35:20.959127 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231611: 924574 keys, 37630368 bytes, temperature: kWarm 2024/11/22-22:35:20.959337 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286120959212, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231611, "file_size": 37630368, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3613581080, "table_properties": {"data_size": 31547888, "index_size": 1758976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25888072, "raw_average_key_size": 28, "raw_value_size": 108329776, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924574, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286115, "oldest_key_time": 0, "file_creation_time": 1732286115, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231611, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:21.012083 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231612.sst) /dev/shm/ToplingTemp/Topling-317308-scqCGy 2024/11/22-22:35:22.611596 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231612.sst): first pass time = 1.60's, 83.914'MB/sec 2024/11/22-22:35:24.210309 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231612.sst): old prealloc_size = 147639500, real_size = 32983664 2024/11/22-22:35:24.211017 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231612.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930161 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.30 avg-zval = 33.31 2024/11/22-22:35:24.211844 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231612.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:25.531383 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231612.sst): second pass time = 1.60's, 67.708'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.32's, 28.077'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2567.749'MB/sec (index lex order gen) rebuild zvType time = 1.06's, 0.219'MB/sec write SST data time = 0.26's, 143.398'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 25735.509'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.708'MB/sec zip pipeline throughput = 67.708'MB/sec entries = 930161 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.30 avg-zval = 33.31 usrkeys = 930161 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.30 avg-zval = 33.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930161 bytes = 3255584 } seq expand size = 6511127 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5130 value = 3.4916 dict = 1.00 all = 3.6186 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 0.344283 GB avg = 0.124 KB (by entry num) total key len = 0.077827 GB avg = 0.028 KB total ukey len = 0.055591 GB avg = 0.020 KB total ukey num = 0.002779536 Billion total entry num = 0.002779536 Billion write speed all = 0.008547779 MB/sec (with seq num) write speed all = 0.008097491 MB/sec (without seq num) 2024/11/22-22:35:25.569267 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231612: 930161 keys, 37091444 bytes, temperature: kWarm 2024/11/22-22:35:25.569349 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286125569306, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231612, "file_size": 37091444, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3613582230, "table_properties": {"data_size": 30981408, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26044508, "raw_average_key_size": 28, "raw_value_size": 108173278, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930161, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286120, "oldest_key_time": 0, "file_creation_time": 1732286120, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231612, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:25.628044 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231613.sst) /dev/shm/ToplingTemp/Topling-317308-sk66qz 2024/11/22-22:35:27.316867 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231613.sst): first pass time = 1.69's, 79.476'MB/sec 2024/11/22-22:35:29.115524 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231613.sst): old prealloc_size = 147639500, real_size = 32767248 2024/11/22-22:35:29.116128 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231613.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924689 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.15 avg-zval = 33.28 2024/11/22-22:35:29.117985 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231613.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:30.587600 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231613.sst): second pass time = 1.80's, 60.253'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.47's, 25.039'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3101.567'MB/sec (index lex order gen) rebuild zvType time = 1.20's, 0.193'MB/sec write SST data time = 0.27's, 135.290'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32634.171'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.253'MB/sec zip pipeline throughput = 60.253'MB/sec entries = 924689 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 33.28 usrkeys = 924689 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 33.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924689 bytes = 3236432 } seq expand size = 6472823 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5139 value = 3.5197 dict = 1.00 all = 3.6413 } Zip/UnZip{ index = 0.0951 value = 0.2841 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 0.459083 GB avg = 0.124 KB (by entry num) total key len = 0.103718 GB avg = 0.028 KB total ukey len = 0.074084 GB avg = 0.020 KB total ukey num = 0.003704225 Billion total entry num = 0.003704225 Billion write speed all = 0.011395615 MB/sec (with seq num) write speed all = 0.010795588 MB/sec (without seq num) 2024/11/22-22:35:30.624256 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231613: 924689 keys, 36859588 bytes, temperature: kWarm 2024/11/22-22:35:30.624315 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286130624283, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231613, "file_size": 36859588, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3435916124, "largest_seqno": 3613583151, "table_properties": {"data_size": 30776896, "index_size": 1758976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25891292, "raw_average_key_size": 28, "raw_value_size": 108326578, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924689, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286125, "oldest_key_time": 0, "file_creation_time": 1732286125, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231613, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:30.661228 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231614.sst) /dev/shm/ToplingTemp/Topling-317308-h1vtPx 2024/11/22-22:35:31.873590 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231614.sst): first pass time = 1.21's, 110.711'MB/sec 2024/11/22-22:35:33.613162 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231614.sst): old prealloc_size = 147639500, real_size = 33600256 2024/11/22-22:35:33.613857 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231614.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924485 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.18 avg-zval = 34.19 2024/11/22-22:35:33.614651 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231614.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:34.812969 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231614.sst): second pass time = 1.73's, 62.800'MB/sec, value only(80.7% of KV) wait indexing time = 0.01's, remap KeyValue time = 1.20's, 31.415'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2609.164'MB/sec (index lex order gen) rebuild zvType time = 0.99's, 0.234'MB/sec write SST data time = 0.21's, 177.439'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 39039.163'MB/sec zip my value time = 1.73's, unzip length = 0.108'GB zip my value throughput = 62.800'MB/sec zip pipeline throughput = 62.800'MB/sec entries = 924485 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.18 avg-zval = 34.19 usrkeys = 924485 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.18 avg-zval = 34.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924485 bytes = 3235712 } seq expand size = 6471395 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5132 value = 3.4271 dict = 1.00 all = 3.5609 } Zip/UnZip{ index = 0.0951 value = 0.2918 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 0.573886 GB avg = 0.124 KB (by entry num) total key len = 0.129604 GB avg = 0.028 KB total ukey len = 0.092574 GB avg = 0.020 KB total ukey num = 0.004628710 Billion total entry num = 0.004628710 Billion write speed all = 0.014243077 MB/sec (with seq num) write speed all = 0.013493363 MB/sec (without seq num) 2024/11/22-22:35:34.847841 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231614: 924485 keys, 37692284 bytes, temperature: kWarm 2024/11/22-22:35:34.847898 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286134847868, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231614, "file_size": 37692284, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3613583021, "table_properties": {"data_size": 31610224, "index_size": 1758720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25885580, "raw_average_key_size": 28, "raw_value_size": 108332230, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924485, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286130, "oldest_key_time": 0, "file_creation_time": 1732286130, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231614, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:34.887709 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231615.sst) /dev/shm/ToplingTemp/Topling-317308-1tkxXz 2024/11/22-22:35:35.925455 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231615.sst): first pass time = 1.04's, 129.342'MB/sec 2024/11/22-22:35:37.352881 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231615.sst): old prealloc_size = 147639500, real_size = 32895072 2024/11/22-22:35:37.353457 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231615.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928593 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.54 avg-zval = 33.27 2024/11/22-22:35:37.354237 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231615.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:38.297647 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231615.sst): second pass time = 1.43's, 75.844'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.039'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.511'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.13's, 287.888'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36483.612'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.844'MB/sec zip pipeline throughput = 75.844'MB/sec entries = 928593 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.54 avg-zval = 33.27 usrkeys = 928593 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.54 avg-zval = 33.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928593 bytes = 3134016 } seq expand size = 6500151 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5136 value = 3.5026 dict = 1.00 all = 3.6391 } Zip/UnZip{ index = 0.0951 value = 0.2855 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 0.688603 GB avg = 0.124 KB (by entry num) total key len = 0.155604 GB avg = 0.028 KB total ukey len = 0.111146 GB avg = 0.020 KB total ukey num = 0.005557303 Billion total entry num = 0.005557303 Billion write speed all = 0.017090889 MB/sec (with seq num) write speed all = 0.016190834 MB/sec (without seq num) 2024/11/22-22:35:38.332954 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231615: 928593 keys, 36882464 bytes, temperature: kWarm 2024/11/22-22:35:38.333014 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286138332983, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231615, "file_size": 36882464, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3522904895, "largest_seqno": 3613581760, "table_properties": {"data_size": 30896272, "index_size": 1766464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26000604, "raw_average_key_size": 28, "raw_value_size": 108217130, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928593, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286134, "oldest_key_time": 0, "file_creation_time": 1732286134, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231615, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:38.371545 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231616.sst) /dev/shm/ToplingTemp/Topling-317308-j4oS8w 2024/11/22-22:35:39.227236 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231616.sst): first pass time = 0.86's, 156.860'MB/sec 2024/11/22-22:35:40.638191 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231616.sst): old prealloc_size = 147639500, real_size = 31451728 2024/11/22-22:35:40.638730 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231616.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926195 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0295 GB avg-val = 116.91 avg-zval = 31.81 2024/11/22-22:35:40.639502 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231616.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:41.505338 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231616.sst): second pass time = 1.41's, 76.783'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 40.995'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.075'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.13's, 280.374'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36770.309'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.783'MB/sec zip pipeline throughput = 76.783'MB/sec entries = 926195 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 31.81 usrkeys = 926195 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 31.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926195 bytes = 3241696 } seq expand size = 6483365 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5131 value = 3.6759 dict = 1.00 all = 3.7757 } Zip/UnZip{ index = 0.0951 value = 0.2720 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 0.803371 GB avg = 0.124 KB (by entry num) total key len = 0.181538 GB avg = 0.028 KB total ukey len = 0.129670 GB avg = 0.020 KB total ukey num = 0.006483498 Billion total entry num = 0.006483498 Billion write speed all = 0.019938073 MB/sec (with seq num) write speed all = 0.018888080 MB/sec (without seq num) 2024/11/22-22:35:41.538091 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231616: 926195 keys, 35548086 bytes, temperature: kWarm 2024/11/22-22:35:41.538156 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286141538122, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231616, "file_size": 35548086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3613582117, "table_properties": {"data_size": 29457984, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933460, "raw_average_key_size": 28, "raw_value_size": 108284362, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926195, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286138, "oldest_key_time": 0, "file_creation_time": 1732286138, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231616, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:41.575274 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231617.sst) /dev/shm/ToplingTemp/Topling-317308-teMsew 2024/11/22-22:35:42.497793 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231617.sst): first pass time = 0.92's, 145.496'MB/sec 2024/11/22-22:35:43.976327 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231617.sst): old prealloc_size = 147639500, real_size = 32982240 2024/11/22-22:35:43.976886 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231617.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924969 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.11 avg-zval = 33.50 2024/11/22-22:35:43.977643 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231617.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:44.884962 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231617.sst): second pass time = 1.48's, 73.295'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.935'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3332.206'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 294.358'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35248.606'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.295'MB/sec zip pipeline throughput = 73.295'MB/sec entries = 924969 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.11 avg-zval = 33.50 usrkeys = 924969 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.11 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924969 bytes = 3353024 } seq expand size = 6474783 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5122 value = 3.4952 dict = 1.00 all = 3.6085 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 0.918165 GB avg = 0.124 KB (by entry num) total key len = 0.207437 GB avg = 0.028 KB total ukey len = 0.148169 GB avg = 0.020 KB total ukey num = 0.007408467 Billion total entry num = 0.007408467 Billion write speed all = 0.022784634 MB/sec (with seq num) write speed all = 0.021584925 MB/sec (without seq num) 2024/11/22-22:35:44.921114 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231617: 924969 keys, 37194982 bytes, temperature: kWarm 2024/11/22-22:35:44.921171 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286144921140, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231617, "file_size": 37194982, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3613583450, "table_properties": {"data_size": 30990992, "index_size": 1759808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899132, "raw_average_key_size": 28, "raw_value_size": 108318634, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924969, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286141, "oldest_key_time": 0, "file_creation_time": 1732286141, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231617, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:44.961053 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231618.sst) /dev/shm/ToplingTemp/Topling-317308-ZzdCcx 2024/11/22-22:35:45.733439 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231618.sst): first pass time = 0.77's, 173.779'MB/sec 2024/11/22-22:35:47.209620 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231618.sst): old prealloc_size = 147639500, real_size = 37207696 2024/11/22-22:35:47.210169 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231618.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926293 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0352 GB avg-val = 116.90 avg-zval = 38.02 2024/11/22-22:35:47.210505 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231618.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:47.889643 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231618.sst): second pass time = 1.48's, 73.383'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.68's, 57.417'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3340.522'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.420'MB/sec write SST data time = 0.13's, 304.214'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36549.384'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.383'MB/sec zip pipeline throughput = 73.383'MB/sec entries = 926293 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.90 avg-zval = 38.02 usrkeys = 926293 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.83 avg-zval = 38.02 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 256319 bytes = 833056 } seq expand size = 1794233 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5134 value = 3.0750 dict = 1.00 all = 3.4376 } Zip/UnZip{ index = 0.0951 value = 0.3252 dict = 1.00 all = 0.2909 } ---------------------------- total value len = 1.028240 GB avg = 0.123 KB (by entry num) total key len = 0.233373 GB avg = 0.028 KB total ukey len = 0.166695 GB avg = 0.020 KB total ukey num = 0.008334760 Billion total entry num = 0.008334760 Billion write speed all = 0.025536259 MB/sec (with seq num) write speed all = 0.024186632 MB/sec (without seq num) 2024/11/22-22:35:47.923704 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231618: 926293 keys, 39044081 bytes, temperature: kWarm 2024/11/22-22:35:47.923766 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286147923734, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231618, "file_size": 39044081, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 35213824, "index_size": 1762112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25936204, "raw_average_key_size": 28, "raw_value_size": 108281534, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926293, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286144, "oldest_key_time": 0, "file_creation_time": 1732286144, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231618, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:47.957431 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231619.sst) /dev/shm/ToplingTemp/Topling-317308-u0Azpy 2024/11/22-22:35:48.775516 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231619.sst): first pass time = 0.82's, 164.071'MB/sec 2024/11/22-22:35:50.315960 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231619.sst): old prealloc_size = 147639500, real_size = 36471936 2024/11/22-22:35:50.316501 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231619.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927084 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.19 2024/11/22-22:35:50.316545 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231619.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:50.954524 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231619.sst): second pass time = 1.54's, 70.310'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.461'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3369.000'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.451'MB/sec write SST data time = 0.12's, 300.544'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36291.300'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.310'MB/sec zip pipeline throughput = 70.310'MB/sec entries = 927084 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 37.19 usrkeys = 927084 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 37.19 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5125 value = 3.1401 dict = 1.00 all = 3.5954 } Zip/UnZip{ index = 0.0951 value = 0.3185 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 1.136500 GB avg = 0.123 KB (by entry num) total key len = 0.259332 GB avg = 0.028 KB total ukey len = 0.185237 GB avg = 0.020 KB total ukey num = 0.009261844 Billion total entry num = 0.009261844 Billion write speed all = 0.028251202 MB/sec (with seq num) write speed all = 0.026751547 MB/sec (without seq num) 2024/11/22-22:35:50.989884 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231619: 927084 keys, 37330058 bytes, temperature: kWarm 2024/11/22-22:35:50.989946 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286150989913, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231619, "file_size": 37330058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34476208, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958352, "raw_average_key_size": 28, "raw_value_size": 108259412, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927084, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286147, "oldest_key_time": 0, "file_creation_time": 1732286147, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231619, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:51.028866 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231620.sst) /dev/shm/ToplingTemp/Topling-317308-N0WRxz 2024/11/22-22:35:51.915633 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231620.sst): first pass time = 0.89's, 151.363'MB/sec 2024/11/22-22:35:53.372815 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231620.sst): old prealloc_size = 147639500, real_size = 36285680 2024/11/22-22:35:53.373362 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231620.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924138 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0343 GB avg-val = 117.24 avg-zval = 37.11 2024/11/22-22:35:53.373719 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231620.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:54.115165 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231620.sst): second pass time = 1.46's, 74.387'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 51.795'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3343.949'MB/sec (index lex order gen) rebuild zvType time = 0.62's, 0.375'MB/sec write SST data time = 0.13's, 305.861'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 26968.200'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.387'MB/sec zip pipeline throughput = 74.387'MB/sec entries = 924138 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.24 avg-zval = 37.11 usrkeys = 924138 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.84 avg-zval = 37.11 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 343425 bytes = 1159072 } seq expand size = 2403975 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5119 value = 3.1590 dict = 1.00 all = 3.4907 } Zip/UnZip{ index = 0.0951 value = 0.3166 dict = 1.00 all = 0.2865 } ---------------------------- total value len = 1.247246 GB avg = 0.122 KB (by entry num) total key len = 0.285207 GB avg = 0.028 KB total ukey len = 0.203720 GB avg = 0.020 KB total ukey num = 0.010185982 Billion total entry num = 0.010185982 Billion write speed all = 0.031014402 MB/sec (with seq num) write speed all = 0.029365219 MB/sec (without seq num) 2024/11/22-22:35:54.153861 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231620: 924138 keys, 38449757 bytes, temperature: kWarm 2024/11/22-22:35:54.153921 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286154153890, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231620, "file_size": 38449757, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3375960067, "table_properties": {"data_size": 34296160, "index_size": 1758272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25875864, "raw_average_key_size": 28, "raw_value_size": 108342004, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924138, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286151, "oldest_key_time": 0, "file_creation_time": 1732286151, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231620, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:54.193065 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231621.sst) /dev/shm/ToplingTemp/Topling-317308-k5wOYv 2024/11/22-22:35:54.996640 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231621.sst): first pass time = 0.80's, 167.034'MB/sec 2024/11/22-22:35:56.336365 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231621.sst): old prealloc_size = 147639500, real_size = 33374384 2024/11/22-22:35:56.336913 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231621.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925049 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.09 avg-zval = 33.93 2024/11/22-22:35:56.338949 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231621.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:35:57.270830 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231621.sst): second pass time = 1.34's, 80.891'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.096'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3342.322'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 293.570'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36303.264'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.891'MB/sec zip pipeline throughput = 80.891'MB/sec entries = 925049 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.09 avg-zval = 33.93 usrkeys = 925049 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.09 avg-zval = 33.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925049 bytes = 3237680 } seq expand size = 6475343 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5119 value = 3.4514 dict = 1.00 all = 3.5822 } Zip/UnZip{ index = 0.0951 value = 0.2897 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 1.362038 GB avg = 0.123 KB (by entry num) total key len = 0.311109 GB avg = 0.028 KB total ukey len = 0.222221 GB avg = 0.020 KB total ukey num = 0.011111031 Billion total entry num = 0.011111031 Billion write speed all = 0.033859645 MB/sec (with seq num) write speed all = 0.032060804 MB/sec (without seq num) 2024/11/22-22:35:57.303411 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231621: 925049 keys, 37467606 bytes, temperature: kWarm 2024/11/22-22:35:57.303474 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286157303441, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231621, "file_size": 37467606, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375960068, "largest_seqno": 3613583115, "table_properties": {"data_size": 31382944, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25901372, "raw_average_key_size": 28, "raw_value_size": 108316486, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925049, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286154, "oldest_key_time": 0, "file_creation_time": 1732286154, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231621, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:35:57.344343 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231622.sst) /dev/shm/ToplingTemp/Topling-317308-Fn0Akx 2024/11/22-22:35:58.168356 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231622.sst): first pass time = 0.82's, 162.891'MB/sec 2024/11/22-22:35:59.507193 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231622.sst): old prealloc_size = 147639500, real_size = 34372736 2024/11/22-22:35:59.507717 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231622.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930556 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.23 avg-zval = 34.77 2024/11/22-22:35:59.509667 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231622.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:36:00.413359 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231622.sst): second pass time = 1.34's, 80.825'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.595'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3560.572'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.300'MB/sec write SST data time = 0.13's, 298.965'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35161.318'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.825'MB/sec zip pipeline throughput = 80.825'MB/sec entries = 930556 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.23 avg-zval = 34.77 usrkeys = 930556 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.23 avg-zval = 34.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930556 bytes = 3373280 } seq expand size = 6513892 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4450 value = 3.3427 dict = 1.00 all = 3.4774 } Zip/UnZip{ index = 0.0957 value = 0.2992 dict = 1.00 all = 0.2876 } ---------------------------- total value len = 1.476714 GB avg = 0.123 KB (by entry num) total key len = 0.337164 GB avg = 0.028 KB total ukey len = 0.240832 GB avg = 0.020 KB total ukey num = 0.012041587 Billion total entry num = 0.012041587 Billion write speed all = 0.036705314 MB/sec (with seq num) write speed all = 0.034755942 MB/sec (without seq num) 2024/11/22-22:36:00.448211 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231622: 930556 keys, 38597230 bytes, temperature: kWarm 2024/11/22-22:36:00.448275 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286160448243, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231622, "file_size": 38597230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3613582512, "table_properties": {"data_size": 32358064, "index_size": 1781824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26055568, "raw_average_key_size": 28, "raw_value_size": 108162232, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930556, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286157, "oldest_key_time": 0, "file_creation_time": 1732286157, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231622, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:36:00.483848 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231623.sst) /dev/shm/ToplingTemp/Topling-317308-rXfbry 2024/11/22-22:36:01.283386 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231623.sst): first pass time = 0.80's, 167.878'MB/sec 2024/11/22-22:36:02.578860 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231623.sst): old prealloc_size = 147639500, real_size = 31245168 2024/11/22-22:36:02.579410 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231623.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923870 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0292 GB avg-val = 117.28 avg-zval = 31.65 2024/11/22-22:36:02.581473 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231623.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:36:03.438278 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231623.sst): second pass time = 1.29's, 83.678'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.117'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3348.767'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 276.802'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35459.638'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.678'MB/sec zip pipeline throughput = 83.678'MB/sec entries = 923870 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.28 avg-zval = 31.65 usrkeys = 923870 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.28 avg-zval = 31.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923870 bytes = 3233552 } seq expand size = 6467090 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4427 value = 3.7049 dict = 1.00 all = 3.7983 } Zip/UnZip{ index = 0.0958 value = 0.2699 dict = 1.00 all = 0.2633 } ---------------------------- total value len = 1.591530 GB avg = 0.123 KB (by entry num) total key len = 0.363033 GB avg = 0.028 KB total ukey len = 0.259309 GB avg = 0.020 KB total ukey num = 0.012965457 Billion total entry num = 0.012965457 Billion write speed all = 0.039549768 MB/sec (with seq num) write speed all = 0.037450963 MB/sec (without seq num) 2024/11/22-22:36:03.471817 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231623: 923870 keys, 35336146 bytes, temperature: kWarm 2024/11/22-22:36:03.471885 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286163471851, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231623, "file_size": 35336146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3406990627, "largest_seqno": 3613582198, "table_properties": {"data_size": 29244640, "index_size": 1769408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868360, "raw_average_key_size": 28, "raw_value_size": 108349476, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923870, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286160, "oldest_key_time": 0, "file_creation_time": 1732286160, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231623, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:36:03.508445 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231624.sst) /dev/shm/ToplingTemp/Topling-317308-y07Ajx 2024/11/22-22:36:04.381842 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231624.sst): first pass time = 0.87's, 153.681'MB/sec 2024/11/22-22:36:05.724079 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231624.sst): old prealloc_size = 147639500, real_size = 33377008 2024/11/22-22:36:05.724587 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231624.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923878 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.28 avg-zval = 33.95 2024/11/22-22:36:05.726649 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231624.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:36:06.668124 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231624.sst): second pass time = 1.34's, 80.764'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.689'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3666.760'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.283'MB/sec write SST data time = 0.13's, 291.401'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 36407.615'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.764'MB/sec zip pipeline throughput = 80.764'MB/sec entries = 923878 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.28 avg-zval = 33.95 usrkeys = 923878 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.28 avg-zval = 33.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923878 bytes = 3233584 } seq expand size = 6467146 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3771 value = 3.4544 dict = 1.00 all = 3.5822 } Zip/UnZip{ index = 0.0964 value = 0.2895 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 1.706347 GB avg = 0.123 KB (by entry num) total key len = 0.388901 GB avg = 0.028 KB total ukey len = 0.277787 GB avg = 0.020 KB total ukey num = 0.013889335 Billion total entry num = 0.013889335 Billion write speed all = 0.042393699 MB/sec (with seq num) write speed all = 0.040145486 MB/sec (without seq num) 2024/11/22-22:36:06.705689 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231624: 923878 keys, 37467786 bytes, temperature: kWarm 2024/11/22-22:36:06.705754 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286166705721, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231624, "file_size": 37467786, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3470221528, "largest_seqno": 3613579895, "table_properties": {"data_size": 31365216, "index_size": 1780608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868584, "raw_average_key_size": 28, "raw_value_size": 108349236, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923878, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286163, "oldest_key_time": 0, "file_creation_time": 1732286163, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231624, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:36:06.745552 140025137890880 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231625.sst) /dev/shm/ToplingTemp/Topling-317308-0eJWRy 2024/11/22-22:36:06.831892 140025137890880 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231625.sst): first pass time = 0.09's, 227.931'MB/sec 2024/11/22-22:36:07.005344 140025137890880 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231625.sst): old prealloc_size = 147639500, real_size = 5669152 2024/11/22-22:36:07.005469 140025137890880 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231625.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 136310 prefix = 4 raw-key = 0.0022 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.0159 GB zip-val = 0.0054 GB avg-val = 116.30 avg-zval = 39.40 2024/11/22-22:36:07.005597 140025137890880 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231625.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.020 GB 2024/11/22-22:36:07.134326 140025137890880 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00/231625.sst): second pass time = 0.17's, 91.688'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.13's, 48.426'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2639.235'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.311'MB/sec write SST data time = 0.02's, 324.391'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.166'MB, throughput = 6084.263'MB/sec zip my value time = 0.17's, unzip length = 0.016'GB zip my value throughput = 91.688'MB/sec zip pipeline throughput = 91.688'MB/sec entries = 136310 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.30 avg-zval = 39.40 usrkeys = 136310 avg-key = 20.00 avg-zkey = 1.94 avg-val = 123.30 avg-zval = 39.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 136310 bytes = 443024 } seq expand size = 954170 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0027 GB value = 0.0159 GB dict = 0.17 MB all = 0.0197 GB } __ZipSize{ index = 0.0003 GB value = 0.0054 GB dict = 0.17 MB all = 0.0062 GB } UnZip/Zip{ index = 10.3265 value = 2.9517 dict = 1.00 all = 3.1495 } Zip/UnZip{ index = 0.0968 value = 0.3388 dict = 1.00 all = 0.3175 } ---------------------------- total value len = 1.723154 GB avg = 0.123 KB (by entry num) total key len = 0.392718 GB avg = 0.028 KB total ukey len = 0.280513 GB avg = 0.020 KB total ukey num = 0.014025645 Billion total entry num = 0.014025645 Billion write speed all = 0.042810583 MB/sec (with seq num) write speed all = 0.040540328 MB/sec (without seq num) 2024/11/22-22:36:07.145811 140025137890880 [compaction_job.cc:1948] [default] [JOB 12905] Generated table #231625: 136310 keys, 6245351 bytes, temperature: kWarm 2024/11/22-22:36:07.145866 140025137890880 EVENT_LOG_v1 {"time_micros": 1732286167145837, "cf_name": "default", "job": 12905, "event": "table_file_creation", "file_number": 231625, "file_size": 6245351, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3575136267, "largest_seqno": 3612466161, "table_properties": {"data_size": 5370896, "index_size": 264000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3816680, "raw_average_key_size": 28, "raw_value_size": 15853028, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 136310, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286166, "oldest_key_time": 0, "file_creation_time": 1732286166, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 231625, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:36:07.157693 140025137890880 [compaction_job.cc:703] job-12905: subcompact[0], size: 0.565864 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12905/att-00 [231610,231611,231612,231613,231614,231615,231616,231617,231618,231619,231620,231621,231622,231623,231624,231625] 2024/11/22-22:36:07.258525 140025137890880 (Original Log Time 2024/11/22-22:35:09.053643) EVENT_LOG_v1 {"time_micros": 1732286109053601, "job": 12905, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [231598], "files_L3": [231533, 231534, 231535, 231536, 231537, 231538, 231539, 231540, 231541, 231542, 231543, 231544, 231545, 231546, 231562], "score": 0.998735, "input_data_size": 589296980, "oldest_snapshot_seqno": 3613614025} 2024/11/22-22:36:07.258528 140025137890880 (Original Log Time 2024/11/22-22:36:07.257712) [compaction_job.cc:2022] [default] [JOB 12905] Compacted 1@2 + 15@3 files to L3 => 565864041 bytes 2024/11/22-22:36:07.258530 140025137890880 (Original Log Time 2024/11/22-22:36:07.258461) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.1 rd, 9.7 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.1, 534.9 +0.0 blob) out(539.7 +0.0 blob), read-write-amplify(40.6) write-amplify(19.9) OK, records in: 14123181, records dropped: 97536 output_compression: Snappy 2024/11/22-22:36:07.258536 140025137890880 (Original Log Time 2024/11/22-22:36:07.258506) EVENT_LOG_v1 {"time_micros": 1732286167258484, "cf": "default", "job": 12905, "event": "compaction_finished", "compaction_time_micros": 58104059, "compaction_time_cpu_micros": 24973965, "output_level": 3, "num_output_files": 16, "total_output_size": 565864041, "num_input_records": 14123181, "num_output_records": 14025645, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 457108568, "file_range_sync_nanos": 0, "file_fsync_nanos": 15220, "file_prepare_write_nanos": 38455, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-22:36:07.380838 140025137890880 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732286108,"executesMs":58332,"compactionJobId":12905,"attempt":0,"compactionInputRawBytes":2046786890,"compactionInputZipBytes":589296980,"compactionOutputRawBytes":0,"compactionOutputZipBytes":565864041,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-94853f800920525da2c9ed5568ed04ad-349b9f590d69e796-00"}}, response = 2024/11/22-22:36:07.380907 140025137890880 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12905/att-00: olev 3, work 58.332 s, result 6.393 ms, install 0.843 ms, input{raw 1.906 GiB zip 561.997 MiB}