2024/11/23-15:23:58.910132 139957618013760 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00: fork to child time = 0.008341 sec 2024/11/23-15:23:58.910885 139957618013760 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00 2024/11/23-15:23:58.917936 139957618013760 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00 2024/11/23-15:23:58.918804 139957618013760 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-15:23:58.919065 139957618013760 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-15:23:58.980034 139957618013760 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29193/att-00/MANIFEST-206347 2024/11/23-15:23:58.995465 139957618013760 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29193/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 266335, last_sequence is 4065345595, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 266195 2024/11/23-15:23:58.995473 139957618013760 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-15:23:58.995582 139957618013760 [version_set.cc:5721] Creating manifest 266335 2024/11/23-15:23:59.032218 139957618013760 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-15:23:59.056512 139957618013760 [compaction_job.cc:2437] [default] [JOB 29193] Compacting 1@2 + 23@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-15:23:59.056528 139957618013760 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21826 Base level 2, inputs: [266322(24MB)], [266274(39MB) 266275(38MB) 266276(36MB) 266277(34MB) 266278(35MB) 266279(34MB) 266280(34MB) 266281(33MB) 266282(34MB) 266283(33MB) 266284(35MB) 266285(38MB) 266286(35MB) 266287(35MB) 266288(33MB) 266289(35MB) 266290(36MB) 266291(35MB) 266292(35MB) 266293(35MB) 266294(36MB) 266295(37MB) 266299(37MB)] 2024/11/23-15:23:59.063296 139957618013760 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:23:59.063321 139957618013760 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:23:59.063337 139957618013760 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:23:59.063342 139957618013760 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:23:59.063352 139957618013760 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:23:59.663644 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266336.sst) /dev/shm/ToplingTemp/Topling-93175-uOxpsD 2024/11/23-15:24:00.550640 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266336.sst): first pass time = 0.89's, 151.326'MB/sec 2024/11/23-15:24:03.865677 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266336.sst): old prealloc_size = 147639500, real_size = 37231312 2024/11/23-15:24:03.866241 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266336.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928265 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0352 GB avg-val = 116.59 avg-zval = 37.94 2024/11/23-15:24:03.867029 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266336.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:04.919370 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266336.sst): second pass time = 3.31's, 32.661'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 39.008'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.114'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.23's, 179.405'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34990.374'MB/sec zip my value time = 3.31's, unzip length = 0.108'GB zip my value throughput = 32.661'MB/sec zip pipeline throughput = 107.502'MB/sec entries = 928265 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.59 avg-zval = 37.94 usrkeys = 928265 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.59 avg-zval = 37.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928265 bytes = 3016880 } seq expand size = 6497855 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.4463 value = 3.0727 dict = 1.00 all = 3.2655 } Zip/UnZip{ index = 0.0957 value = 0.3254 dict = 1.00 all = 0.3062 } ---------------------------- total value len = 8.963344 GB avg = 0.168 KB (by entry num) total key len = 1.466813 GB avg = 0.028 KB total ukey len = 1.016443 GB avg = 0.015 KB total ukey num = 0.066478700 Billion total entry num = 0.053286429 Billion write speed all = 0.447018754 MB/sec (with seq num) write speed all = 0.428748629 MB/sec (without seq num) 2024/11/23-15:24:04.972736 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266336: 928265 keys, 41101468 bytes, temperature: kWarm 2024/11/23-15:24:04.972864 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346644972787, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266336, "file_size": 41101468, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777911, "largest_seqno": 4065296300, "table_properties": {"data_size": 35221824, "index_size": 1777216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25991420, "raw_average_key_size": 28, "raw_value_size": 108226438, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928265, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346639, "oldest_key_time": 0, "file_creation_time": 1732346639, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266336, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:05.036886 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266337.sst) /dev/shm/ToplingTemp/Topling-93175-NS64FF 2024/11/23-15:24:06.435267 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266337.sst): first pass time = 1.40's, 95.985'MB/sec 2024/11/23-15:24:08.814760 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266337.sst): old prealloc_size = 147639500, real_size = 36177888 2024/11/23-15:24:08.815393 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266337.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925398 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0342 GB avg-val = 117.04 avg-zval = 36.95 2024/11/23-15:24:08.816251 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266337.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:09.891712 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266337.sst): second pass time = 2.38's, 45.538'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 37.507'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2897.067'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.275'MB/sec write SST data time = 0.24's, 171.581'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30834.332'MB/sec zip my value time = 2.38's, unzip length = 0.108'GB zip my value throughput = 45.538'MB/sec zip pipeline throughput = 97.807'MB/sec entries = 925398 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.04 avg-zval = 36.95 usrkeys = 925398 avg-key = 20.00 avg-zkey = 1.89 avg-val = 124.04 avg-zval = 36.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925398 bytes = 3354576 } seq expand size = 6477786 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0342 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.5832 value = 3.1671 dict = 1.00 all = 3.3229 } Zip/UnZip{ index = 0.0945 value = 0.3157 dict = 1.00 all = 0.3009 } ---------------------------- total value len = 9.078129 GB avg = 0.167 KB (by entry num) total key len = 1.492724 GB avg = 0.028 KB total ukey len = 1.034951 GB avg = 0.015 KB total ukey num = 0.067404098 Billion total entry num = 0.054211827 Billion write speed all = 0.452952202 MB/sec (with seq num) write speed all = 0.434368750 MB/sec (without seq num) 2024/11/23-15:24:09.946561 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266337: 925398 keys, 40391662 bytes, temperature: kWarm 2024/11/23-15:24:09.946659 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346649946604, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266337, "file_size": 40391662, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3736072251, "largest_seqno": 4065295770, "table_properties": {"data_size": 34197584, "index_size": 1748800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911144, "raw_average_key_size": 28, "raw_value_size": 108306696, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925398, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346645, "oldest_key_time": 0, "file_creation_time": 1732346645, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266337, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:10.016336 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266338.sst) /dev/shm/ToplingTemp/Topling-93175-dEeXtD 2024/11/23-15:24:11.364542 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266338.sst): first pass time = 1.35's, 99.556'MB/sec 2024/11/23-15:24:13.396534 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266338.sst): old prealloc_size = 147639500, real_size = 33584992 2024/11/23-15:24:13.397180 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266338.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931817 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0316 GB avg-val = 116.04 avg-zval = 33.86 2024/11/23-15:24:13.398082 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266338.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:14.478570 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266338.sst): second pass time = 2.03's, 53.249'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 35.056'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2898.531'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.275'MB/sec write SST data time = 0.23's, 162.280'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 43455.718'MB/sec zip my value time = 2.03's, unzip length = 0.108'GB zip my value throughput = 53.249'MB/sec zip pipeline throughput = 85.375'MB/sec entries = 931817 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.04 avg-zval = 33.86 usrkeys = 931817 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.04 avg-zval = 33.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931817 bytes = 3494336 } seq expand size = 6522719 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.3771 value = 3.4265 dict = 1.00 all = 3.5385 } Zip/UnZip{ index = 0.0964 value = 0.2918 dict = 1.00 all = 0.2826 } ---------------------------- total value len = 9.192778 GB avg = 0.167 KB (by entry num) total key len = 1.518815 GB avg = 0.028 KB total ukey len = 1.053588 GB avg = 0.015 KB total ukey num = 0.068335915 Billion total entry num = 0.055143644 Billion write speed all = 0.458892624 MB/sec (with seq num) write speed all = 0.439993465 MB/sec (without seq num) 2024/11/23-15:24:14.528062 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266338: 931817 keys, 37930246 bytes, temperature: kWarm 2024/11/23-15:24:14.528148 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346654528103, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266338, "file_size": 37930246, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776149, "table_properties": {"data_size": 31555920, "index_size": 1795904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26090876, "raw_average_key_size": 28, "raw_value_size": 108126926, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931817, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346650, "oldest_key_time": 0, "file_creation_time": 1732346650, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266338, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:14.578232 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266339.sst) /dev/shm/ToplingTemp/Topling-93175-v5qWqE 2024/11/23-15:24:15.767137 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266339.sst): first pass time = 1.19's, 112.896'MB/sec 2024/11/23-15:24:17.362164 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266339.sst): old prealloc_size = 147639500, real_size = 31417440 2024/11/23-15:24:17.362753 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266339.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931619 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0294 GB avg-val = 116.07 avg-zval = 31.57 2024/11/23-15:24:17.363567 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266339.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:18.361711 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266339.sst): second pass time = 1.59's, 67.827'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 35.663'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3126.289'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.304'MB/sec write SST data time = 0.23's, 153.939'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38294.331'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.827'MB/sec zip pipeline throughput = 100.385'MB/sec entries = 931619 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.07 avg-zval = 31.57 usrkeys = 931619 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.07 avg-zval = 31.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931619 bytes = 3377136 } seq expand size = 6521333 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5143 value = 3.6764 dict = 1.00 all = 3.7653 } Zip/UnZip{ index = 0.0951 value = 0.2720 dict = 1.00 all = 0.2656 } ---------------------------- total value len = 9.307432 GB avg = 0.166 KB (by entry num) total key len = 1.544900 GB avg = 0.028 KB total ukey len = 1.072220 GB avg = 0.015 KB total ukey num = 0.069267534 Billion total entry num = 0.056075263 Billion write speed all = 0.464844662 MB/sec (with seq num) write speed all = 0.445629410 MB/sec (without seq num) 2024/11/23-15:24:18.406552 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266339: 931619 keys, 35645502 bytes, temperature: kWarm 2024/11/23-15:24:18.406661 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346658406595, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266339, "file_size": 35645502, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3730964986, "largest_seqno": 4012776165, "table_properties": {"data_size": 29412240, "index_size": 1772096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26085332, "raw_average_key_size": 28, "raw_value_size": 108132434, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931619, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346654, "oldest_key_time": 0, "file_creation_time": 1732346654, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266339, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:18.472159 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266340.sst) /dev/shm/ToplingTemp/Topling-93175-E1BUYD 2024/11/23-15:24:19.504207 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266340.sst): first pass time = 1.03's, 130.057'MB/sec 2024/11/23-15:24:21.009101 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266340.sst): old prealloc_size = 147639500, real_size = 32539920 2024/11/23-15:24:21.009666 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266340.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930633 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.22 avg-zval = 32.81 2024/11/23-15:24:21.010452 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266340.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:22.082902 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266340.sst): second pass time = 1.50's, 71.915'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 34.130'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3256.738'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.269'MB/sec write SST data time = 0.21's, 175.747'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 42253.390'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 71.915'MB/sec zip pipeline throughput = 71.915'MB/sec entries = 930633 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 32.81 usrkeys = 930633 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 32.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930633 bytes = 3257232 } seq expand size = 6514431 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5138 value = 3.5420 dict = 1.00 all = 3.6623 } Zip/UnZip{ index = 0.0951 value = 0.2823 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 9.422106 GB avg = 0.165 KB (by entry num) total key len = 1.570958 GB avg = 0.028 KB total ukey len = 1.090833 GB avg = 0.016 KB total ukey num = 0.070198167 Billion total entry num = 0.057005896 Billion write speed all = 0.470797691 MB/sec (with seq num) write speed all = 0.451266653 MB/sec (without seq num) 2024/11/23-15:24:22.128960 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266340: 930633 keys, 36648542 bytes, temperature: kWarm 2024/11/23-15:24:22.129044 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346662129007, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266340, "file_size": 36648542, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3781087153, "largest_seqno": 4012776176, "table_properties": {"data_size": 30536768, "index_size": 1770304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26057724, "raw_average_key_size": 28, "raw_value_size": 108160066, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930633, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346658, "oldest_key_time": 0, "file_creation_time": 1732346658, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266340, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:22.169609 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266341.sst) /dev/shm/ToplingTemp/Topling-93175-F91KxD 2024/11/23-15:24:23.067183 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266341.sst): first pass time = 0.90's, 149.542'MB/sec 2024/11/23-15:24:24.601094 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266341.sst): old prealloc_size = 147639500, real_size = 32565376 2024/11/23-15:24:24.601614 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266341.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933724 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.74 avg-zval = 32.72 2024/11/23-15:24:24.602227 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266341.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:25.611742 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266341.sst): second pass time = 1.53's, 70.490'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 35.427'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3506.536'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.271'MB/sec write SST data time = 0.15's, 242.220'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34471.980'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.490'MB/sec zip pipeline throughput = 70.490'MB/sec entries = 933724 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.74 avg-zval = 32.72 usrkeys = 933724 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.71 avg-zval = 32.72 TagRS{ kinds = RS_Key0_TagN bytes = 146000 } TagArray{ size = 662782 bytes = 2236912 } seq expand size = 4639474 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5130 value = 3.5370 dict = 1.00 all = 3.7488 } Zip/UnZip{ index = 0.0951 value = 0.2827 dict = 1.00 all = 0.2668 } ---------------------------- total value len = 9.534819 GB avg = 0.165 KB (by entry num) total key len = 1.597102 GB avg = 0.028 KB total ukey len = 1.109507 GB avg = 0.016 KB total ukey num = 0.071131891 Billion total entry num = 0.057939620 Billion write speed all = 0.476672464 MB/sec (with seq num) write speed all = 0.456824519 MB/sec (without seq num) 2024/11/23-15:24:25.658740 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266341: 933724 keys, 35803070 bytes, temperature: kWarm 2024/11/23-15:24:25.658808 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346665658773, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266341, "file_size": 35803070, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776192, "table_properties": {"data_size": 30555440, "index_size": 1776320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26144272, "raw_average_key_size": 28, "raw_value_size": 108073544, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933724, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346662, "oldest_key_time": 0, "file_creation_time": 1732346662, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266341, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:25.696428 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266342.sst) /dev/shm/ToplingTemp/Topling-93175-z6aLBF 2024/11/23-15:24:26.601788 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266342.sst): first pass time = 0.91's, 148.254'MB/sec 2024/11/23-15:24:28.199284 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266342.sst): old prealloc_size = 147639500, real_size = 34979792 2024/11/23-15:24:28.199962 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266342.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081577 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.09 avg-zval = 30.19 2024/11/23-15:24:28.200032 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266342.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:29.066597 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266342.sst): second pass time = 1.60's, 65.098'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.225'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3013.181'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.372'MB/sec write SST data time = 0.14's, 256.148'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.034'MB, throughput = 35316.438'MB/sec zip my value time = 1.60's, unzip length = 0.104'GB zip my value throughput = 65.098'MB/sec zip pipeline throughput = 65.098'MB/sec entries = 1081577 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.19 usrkeys = 1081577 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.19 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5163 value = 3.1830 dict = 1.00 all = 3.7539 } Zip/UnZip{ index = 0.0951 value = 0.3142 dict = 1.00 all = 0.2664 } ---------------------------- total value len = 9.638753 GB avg = 0.163 KB (by entry num) total key len = 1.627386 GB avg = 0.028 KB total ukey len = 1.131139 GB avg = 0.016 KB total ukey num = 0.072213468 Billion total entry num = 0.059021197 Billion write speed all = 0.482348356 MB/sec (with seq num) write speed all = 0.462132893 MB/sec (without seq num) 2024/11/23-15:24:29.112733 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266342: 1081577 keys, 35754288 bytes, temperature: kWarm 2024/11/23-15:24:29.112800 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346669112765, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266342, "file_size": 35754288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32652224, "index_size": 2056960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30284156, "raw_average_key_size": 28, "raw_value_size": 103933631, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081577, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346665, "oldest_key_time": 0, "file_creation_time": 1732346665, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266342, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:29.157875 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266343.sst) /dev/shm/ToplingTemp/Topling-93175-qWjr3D 2024/11/23-15:24:30.023839 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266343.sst): first pass time = 0.87's, 155.003'MB/sec 2024/11/23-15:24:31.539919 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266343.sst): old prealloc_size = 147639500, real_size = 31335744 2024/11/23-15:24:31.540611 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266343.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926246 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0293 GB avg-val = 116.91 avg-zval = 31.68 2024/11/23-15:24:31.541566 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266343.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:32.566736 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266343.sst): second pass time = 1.52's, 71.472'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 34.747'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2610.567'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.260'MB/sec write SST data time = 0.13's, 264.386'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 39316.280'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.472'MB/sec zip pipeline throughput = 71.472'MB/sec entries = 926246 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 31.68 usrkeys = 926246 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.63 avg-zval = 31.68 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 890362 bytes = 3338880 } seq expand size = 6232534 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5137 value = 3.6904 dict = 1.00 all = 3.7619 } Zip/UnZip{ index = 0.0951 value = 0.2710 dict = 1.00 all = 0.2658 } ---------------------------- total value len = 9.753269 GB avg = 0.163 KB (by entry num) total key len = 1.653321 GB avg = 0.028 KB total ukey len = 1.149664 GB avg = 0.016 KB total ukey num = 0.073139714 Billion total entry num = 0.059947443 Billion write speed all = 0.488288420 MB/sec (with seq num) write speed all = 0.467758783 MB/sec (without seq num) 2024/11/23-15:24:32.640867 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266343: 926246 keys, 35677946 bytes, temperature: kWarm 2024/11/23-15:24:32.640919 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346672640886, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266343, "file_size": 35677946, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775885, "table_properties": {"data_size": 29342000, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934888, "raw_average_key_size": 28, "raw_value_size": 108282852, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926246, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346669, "oldest_key_time": 0, "file_creation_time": 1732346669, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266343, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:32.697656 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266344.sst) /dev/shm/ToplingTemp/Topling-93175-oBJaYF 2024/11/23-15:24:33.611292 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266344.sst): first pass time = 0.91's, 146.911'MB/sec 2024/11/23-15:24:35.168359 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266344.sst): old prealloc_size = 147639500, real_size = 32347248 2024/11/23-15:24:35.169083 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266344.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930646 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.22 avg-zval = 32.61 2024/11/23-15:24:35.169962 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266344.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:36.270045 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266344.sst): second pass time = 1.56's, 69.500'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 33.197'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2531.033'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.247'MB/sec write SST data time = 0.16's, 228.585'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34272.534'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.500'MB/sec zip pipeline throughput = 69.500'MB/sec entries = 930646 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 32.61 usrkeys = 930646 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 32.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930646 bytes = 3373600 } seq expand size = 6514522 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5132 value = 3.5645 dict = 1.00 all = 3.6700 } Zip/UnZip{ index = 0.0951 value = 0.2805 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 9.867943 GB avg = 0.162 KB (by entry num) total key len = 1.679379 GB avg = 0.028 KB total ukey len = 1.168277 GB avg = 0.016 KB total ukey num = 0.074070360 Billion total entry num = 0.060878089 Billion write speed all = 0.494234476 MB/sec (with seq num) write speed all = 0.473389435 MB/sec (without seq num) 2024/11/23-15:24:36.315253 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266344: 930646 keys, 36572006 bytes, temperature: kWarm 2024/11/23-15:24:36.315324 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346676315288, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266344, "file_size": 36572006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775902, "table_properties": {"data_size": 30343968, "index_size": 1770432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058088, "raw_average_key_size": 28, "raw_value_size": 108159664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930646, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346672, "oldest_key_time": 0, "file_creation_time": 1732346672, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266344, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:36.356283 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266345.sst) /dev/shm/ToplingTemp/Topling-93175-NupLND 2024/11/23-15:24:37.410415 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266345.sst): first pass time = 1.05's, 127.331'MB/sec 2024/11/23-15:24:38.725502 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266345.sst): old prealloc_size = 147639500, real_size = 32209616 2024/11/23-15:24:38.726091 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266345.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 115.97 avg-zval = 32.39 2024/11/23-15:24:38.726904 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266345.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:39.617296 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266345.sst): second pass time = 1.31's, 82.275'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.732'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3127.891'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.13's, 279.476'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34908.249'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.275'MB/sec zip pipeline throughput = 82.275'MB/sec entries = 932240 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.97 avg-zval = 32.39 usrkeys = 932240 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.97 avg-zval = 32.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932240 bytes = 3262848 } seq expand size = 6525680 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4433 value = 3.5810 dict = 1.00 all = 3.6951 } Zip/UnZip{ index = 0.0958 value = 0.2792 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 9.982583 GB avg = 0.162 KB (by entry num) total key len = 1.705482 GB avg = 0.028 KB total ukey len = 1.186921 GB avg = 0.016 KB total ukey num = 0.075002600 Billion total entry num = 0.061810329 Billion write speed all = 0.500186748 MB/sec (with seq num) write speed all = 0.479025533 MB/sec (without seq num) 2024/11/23-15:24:39.662671 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266345: 932240 keys, 36323126 bytes, temperature: kWarm 2024/11/23-15:24:39.662736 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346679662704, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266345, "file_size": 36323126, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 30191040, "index_size": 1785344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102720, "raw_average_key_size": 28, "raw_value_size": 108115060, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346676, "oldest_key_time": 0, "file_creation_time": 1732346676, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266345, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:39.701190 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266346.sst) /dev/shm/ToplingTemp/Topling-93175-NhHdMG 2024/11/23-15:24:40.609748 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266346.sst): first pass time = 0.91's, 147.732'MB/sec 2024/11/23-15:24:41.949738 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266346.sst): old prealloc_size = 147639500, real_size = 33038960 2024/11/23-15:24:41.950294 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266346.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925338 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.05 avg-zval = 33.54 2024/11/23-15:24:41.951136 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266346.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:42.807151 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266346.sst): second pass time = 1.34's, 80.873'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.578'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3307.982'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 286.973'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31492.887'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.873'MB/sec zip pipeline throughput = 80.873'MB/sec entries = 925338 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.05 avg-zval = 33.54 usrkeys = 925338 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.05 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925338 bytes = 3470032 } seq expand size = 6477366 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4446 value = 3.4898 dict = 1.00 all = 3.5922 } Zip/UnZip{ index = 0.0957 value = 0.2865 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 10.097369 GB avg = 0.161 KB (by entry num) total key len = 1.731391 GB avg = 0.028 KB total ukey len = 1.205428 GB avg = 0.016 KB total ukey num = 0.075927938 Billion total entry num = 0.062735667 Billion write speed all = 0.506138656 MB/sec (with seq num) write speed all = 0.484663577 MB/sec (without seq num) 2024/11/23-15:24:42.853513 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266346: 925338 keys, 37363804 bytes, temperature: kWarm 2024/11/23-15:24:42.853564 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346682853533, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266346, "file_size": 37363804, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775935, "table_properties": {"data_size": 31035552, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25909464, "raw_average_key_size": 28, "raw_value_size": 108308324, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925338, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346679, "oldest_key_time": 0, "file_creation_time": 1732346679, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266346, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:42.894044 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266347.sst) /dev/shm/ToplingTemp/Topling-93175-8ps1nE 2024/11/23-15:24:43.770820 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266347.sst): first pass time = 0.88's, 153.087'MB/sec 2024/11/23-15:24:45.671011 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266347.sst): old prealloc_size = 147639500, real_size = 35563904 2024/11/23-15:24:45.671542 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266347.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928785 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.51 avg-zval = 36.11 2024/11/23-15:24:45.672333 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266347.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:46.619226 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266347.sst): second pass time = 1.90's, 56.970'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 41.957'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3512.213'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 313.010'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32036.927'MB/sec zip my value time = 1.90's, unzip length = 0.108'GB zip my value throughput = 56.970'MB/sec zip pipeline throughput = 126.270'MB/sec entries = 928785 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.51 avg-zval = 36.11 usrkeys = 928785 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.51 avg-zval = 36.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928785 bytes = 3366864 } seq expand size = 6501495 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.3770 value = 3.2262 dict = 1.00 all = 3.3737 } Zip/UnZip{ index = 0.0964 value = 0.3100 dict = 1.00 all = 0.2964 } ---------------------------- total value len = 10.212082 GB avg = 0.160 KB (by entry num) total key len = 1.757397 GB avg = 0.028 KB total ukey len = 1.224004 GB avg = 0.016 KB total ukey num = 0.076856723 Billion total entry num = 0.063664452 Billion write speed all = 0.512076339 MB/sec (with seq num) write speed all = 0.490286881 MB/sec (without seq num) 2024/11/23-15:24:46.672962 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266347: 928785 keys, 39783660 bytes, temperature: kWarm 2024/11/23-15:24:46.673030 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346686672998, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266347, "file_size": 39783660, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33541424, "index_size": 1790080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005980, "raw_average_key_size": 28, "raw_value_size": 108211766, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928785, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346682, "oldest_key_time": 0, "file_creation_time": 1732346682, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266347, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:46.708700 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266348.sst) /dev/shm/ToplingTemp/Topling-93175-jrijbF 2024/11/23-15:24:47.711568 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266348.sst): first pass time = 1.00's, 133.839'MB/sec 2024/11/23-15:24:49.032958 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266348.sst): old prealloc_size = 147639500, real_size = 32794960 2024/11/23-15:24:49.033516 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266348.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928520 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.55 avg-zval = 33.17 2024/11/23-15:24:49.034361 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266348.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:49.870427 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266348.sst): second pass time = 1.32's, 81.947'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.337'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3444.028'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 289.579'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32640.827'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.947'MB/sec zip pipeline throughput = 81.947'MB/sec entries = 928520 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 33.17 usrkeys = 928520 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 33.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928520 bytes = 3481968 } seq expand size = 6499640 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5143 value = 3.5140 dict = 1.00 all = 3.6148 } Zip/UnZip{ index = 0.0951 value = 0.2846 dict = 1.00 all = 0.2766 } ---------------------------- total value len = 10.326801 GB avg = 0.160 KB (by entry num) total key len = 1.783396 GB avg = 0.028 KB total ukey len = 1.242574 GB avg = 0.016 KB total ukey num = 0.077785243 Billion total entry num = 0.064592972 Billion write speed all = 0.518024440 MB/sec (with seq num) write speed all = 0.495920266 MB/sec (without seq num) 2024/11/23-15:24:49.918614 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266348: 928520 keys, 37130204 bytes, temperature: kWarm 2024/11/23-15:24:49.918688 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346689918652, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266348, "file_size": 37130204, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30796416, "index_size": 1766208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25998560, "raw_average_key_size": 28, "raw_value_size": 108219308, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928520, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346686, "oldest_key_time": 0, "file_creation_time": 1732346686, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266348, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:49.955399 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266349.sst) /dev/shm/ToplingTemp/Topling-93175-l7AW3C 2024/11/23-15:24:50.987372 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266349.sst): first pass time = 1.03's, 130.064'MB/sec 2024/11/23-15:24:52.319532 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266349.sst): old prealloc_size = 147639500, real_size = 34766144 2024/11/23-15:24:52.320141 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266349.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926512 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0328 GB avg-val = 116.86 avg-zval = 35.36 2024/11/23-15:24:52.321702 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266349.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:53.141392 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266349.sst): second pass time = 1.33's, 81.355'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 46.916'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3032.029'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 293.667'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36638.364'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.355'MB/sec zip pipeline throughput = 81.355'MB/sec entries = 926512 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.86 avg-zval = 35.36 usrkeys = 926512 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.48 avg-zval = 35.36 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 743515 bytes = 2788192 } seq expand size = 5204605 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4454 value = 3.3051 dict = 1.00 all = 3.4810 } Zip/UnZip{ index = 0.0957 value = 0.3026 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 10.440281 GB avg = 0.159 KB (by entry num) total key len = 1.809338 GB avg = 0.028 KB total ukey len = 1.261104 GB avg = 0.016 KB total ukey num = 0.078711755 Billion total entry num = 0.065519484 Billion write speed all = 0.523915053 MB/sec (with seq num) write speed all = 0.501496956 MB/sec (without seq num) 2024/11/23-15:24:53.189684 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266349: 926512 keys, 38557574 bytes, temperature: kWarm 2024/11/23-15:24:53.189754 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346693189718, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266349, "file_size": 38557574, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 32760304, "index_size": 1774016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942336, "raw_average_key_size": 28, "raw_value_size": 108275480, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926512, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346689, "oldest_key_time": 0, "file_creation_time": 1732346689, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266349, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:53.227899 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266350.sst) /dev/shm/ToplingTemp/Topling-93175-U32dIC 2024/11/23-15:24:54.172750 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266350.sst): first pass time = 0.94's, 142.059'MB/sec 2024/11/23-15:24:55.487429 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266350.sst): old prealloc_size = 147639500, real_size = 32055104 2024/11/23-15:24:55.488051 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266350.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 115.79 avg-zval = 32.16 2024/11/23-15:24:55.489503 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266350.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:56.339138 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266350.sst): second pass time = 1.31's, 82.258'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.609'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2993.776'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 285.922'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38031.546'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.258'MB/sec zip pipeline throughput = 82.258'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.79 avg-zval = 32.16 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.79 avg-zval = 32.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.3780 value = 3.6000 dict = 1.00 all = 3.6986 } Zip/UnZip{ index = 0.0964 value = 0.2778 dict = 1.00 all = 0.2704 } ---------------------------- total value len = 10.554897 GB avg = 0.159 KB (by entry num) total key len = 1.835475 GB avg = 0.028 KB total ukey len = 1.279774 GB avg = 0.016 KB total ukey num = 0.079645212 Billion total entry num = 0.066452941 Billion write speed all = 0.529862536 MB/sec (with seq num) write speed all = 0.507128158 MB/sec (without seq num) 2024/11/23-15:24:56.384033 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266350: 933457 keys, 36289222 bytes, temperature: kWarm 2024/11/23-15:24:56.384098 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346696384063, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266350, "file_size": 36289222, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 30022640, "index_size": 1798912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346693, "oldest_key_time": 0, "file_creation_time": 1732346693, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266350, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:56.438926 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266351.sst) /dev/shm/ToplingTemp/Topling-93175-eySaRC 2024/11/23-15:24:57.364110 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266351.sst): first pass time = 0.93's, 145.078'MB/sec 2024/11/23-15:24:58.672609 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266351.sst): old prealloc_size = 147639500, real_size = 33183168 2024/11/23-15:24:58.673224 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266351.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931549 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 116.08 avg-zval = 33.46 2024/11/23-15:24:58.674019 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266351.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:24:59.515721 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266351.sst): second pass time = 1.31's, 82.692'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.374'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3096.758'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 298.399'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34773.276'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.692'MB/sec zip pipeline throughput = 82.692'MB/sec entries = 931549 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.08 avg-zval = 33.46 usrkeys = 931549 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.08 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931549 bytes = 3376880 } seq expand size = 6520843 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4464 value = 3.4696 dict = 1.00 all = 3.5877 } Zip/UnZip{ index = 0.0957 value = 0.2882 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 10.669552 GB avg = 0.158 KB (by entry num) total key len = 1.861558 GB avg = 0.028 KB total ukey len = 1.298405 GB avg = 0.016 KB total ukey num = 0.080576761 Billion total entry num = 0.067384490 Billion write speed all = 0.535808302 MB/sec (with seq num) write speed all = 0.512758361 MB/sec (without seq num) 2024/11/23-15:24:59.562207 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266351: 931549 keys, 37410846 bytes, temperature: kWarm 2024/11/23-15:24:59.562276 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346699562240, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266351, "file_size": 37410846, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 31166640, "index_size": 1783488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26083372, "raw_average_key_size": 28, "raw_value_size": 108134358, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931549, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346696, "oldest_key_time": 0, "file_creation_time": 1732346696, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266351, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:24:59.599473 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266352.sst) /dev/shm/ToplingTemp/Topling-93175-f8GH6D 2024/11/23-15:25:00.651313 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266352.sst): first pass time = 1.05's, 127.608'MB/sec 2024/11/23-15:25:01.987751 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266352.sst): old prealloc_size = 147639500, real_size = 34730320 2024/11/23-15:25:01.988322 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266352.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 117.01 avg-zval = 35.36 2024/11/23-15:25:01.989883 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266352.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:25:02.831479 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266352.sst): second pass time = 1.34's, 81.086'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.291'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.927'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 307.908'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35552.784'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.086'MB/sec zip pipeline throughput = 81.086'MB/sec entries = 925568 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.01 avg-zval = 35.36 usrkeys = 925568 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.01 avg-zval = 35.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925568 bytes = 3470896 } seq expand size = 6478976 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.4415 value = 3.3094 dict = 1.00 all = 3.4366 } Zip/UnZip{ index = 0.0958 value = 0.3022 dict = 1.00 all = 0.2910 } ---------------------------- total value len = 10.784333 GB avg = 0.158 KB (by entry num) total key len = 1.887474 GB avg = 0.028 KB total ukey len = 1.316916 GB avg = 0.016 KB total ukey num = 0.081502329 Billion total entry num = 0.068310058 Billion write speed all = 0.541747440 MB/sec (with seq num) write speed all = 0.518384205 MB/sec (without seq num) 2024/11/23-15:25:02.880770 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266352: 925568 keys, 39055946 bytes, temperature: kWarm 2024/11/23-15:25:02.880840 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346702880805, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266352, "file_size": 39055946, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32725888, "index_size": 1772864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915904, "raw_average_key_size": 28, "raw_value_size": 108301824, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346699, "oldest_key_time": 0, "file_creation_time": 1732346699, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266352, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:25:02.919252 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266353.sst) /dev/shm/ToplingTemp/Topling-93175-Fw7TpG 2024/11/23-15:25:03.941641 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266353.sst): first pass time = 1.02's, 131.283'MB/sec 2024/11/23-15:25:05.229147 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266353.sst): old prealloc_size = 147639500, real_size = 32518064 2024/11/23-15:25:05.229760 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266353.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932485 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.94 avg-zval = 32.72 2024/11/23-15:25:05.231273 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266353.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:25:06.086397 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266353.sst): second pass time = 1.29's, 84.023'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.870'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3044.747'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 291.416'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29496.169'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.023'MB/sec zip pipeline throughput = 84.023'MB/sec entries = 932485 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.94 avg-zval = 32.72 usrkeys = 932485 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.94 avg-zval = 32.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932485 bytes = 3380272 } seq expand size = 6527395 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5131 value = 3.5433 dict = 1.00 all = 3.6523 } Zip/UnZip{ index = 0.0951 value = 0.2822 dict = 1.00 all = 0.2738 } ---------------------------- total value len = 10.898968 GB avg = 0.157 KB (by entry num) total key len = 1.913584 GB avg = 0.028 KB total ukey len = 1.335566 GB avg = 0.016 KB total ukey num = 0.082434814 Billion total entry num = 0.069242543 Billion write speed all = 0.547688388 MB/sec (with seq num) write speed all = 0.524009523 MB/sec (without seq num) 2024/11/23-15:25:06.133146 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266353: 932485 keys, 36749082 bytes, temperature: kWarm 2024/11/23-15:25:06.133217 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346706133182, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266353, "file_size": 36749082, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 30510816, "index_size": 1773952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26109580, "raw_average_key_size": 28, "raw_value_size": 108108154, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932485, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346702, "oldest_key_time": 0, "file_creation_time": 1732346702, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266353, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:25:06.173297 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266354.sst) /dev/shm/ToplingTemp/Topling-93175-iUqJRE 2024/11/23-15:25:07.207337 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266354.sst): first pass time = 1.03's, 129.804'MB/sec 2024/11/23-15:25:08.556320 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266354.sst): old prealloc_size = 147639500, real_size = 34171616 2024/11/23-15:25:08.556855 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266354.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923008 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0322 GB avg-val = 117.41 avg-zval = 34.87 2024/11/23-15:25:08.558229 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266354.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:25:09.402830 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266354.sst): second pass time = 1.35's, 80.410'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.471'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3453.642'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 306.211'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32887.634'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.410'MB/sec zip pipeline throughput = 80.410'MB/sec entries = 923008 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.41 avg-zval = 34.87 usrkeys = 923008 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.41 avg-zval = 34.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923008 bytes = 3461296 } seq expand size = 6461056 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5132 value = 3.3672 dict = 1.00 all = 3.4870 } Zip/UnZip{ index = 0.0951 value = 0.2970 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 11.013803 GB avg = 0.157 KB (by entry num) total key len = 1.939428 GB avg = 0.028 KB total ukey len = 1.354026 GB avg = 0.016 KB total ukey num = 0.083357822 Billion total entry num = 0.070165551 Billion write speed all = 0.553623397 MB/sec (with seq num) write speed all = 0.529632291 MB/sec (without seq num) 2024/11/23-15:25:09.449942 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266354: 923008 keys, 38490876 bytes, temperature: kWarm 2024/11/23-15:25:09.450027 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346709449991, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266354, "file_size": 38490876, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 32184784, "index_size": 1755904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844224, "raw_average_key_size": 28, "raw_value_size": 108373600, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346706, "oldest_key_time": 0, "file_creation_time": 1732346706, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266354, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:25:09.487383 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266355.sst) /dev/shm/ToplingTemp/Topling-93175-H3x35G 2024/11/23-15:25:10.506585 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266355.sst): first pass time = 1.02's, 131.694'MB/sec 2024/11/23-15:25:11.812095 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266355.sst): old prealloc_size = 147639500, real_size = 33153072 2024/11/23-15:25:11.812638 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266355.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929529 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.39 avg-zval = 33.50 2024/11/23-15:25:11.813451 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266355.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:25:12.667966 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266355.sst): second pass time = 1.30's, 82.949'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.806'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3413.063'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 301.620'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36278.050'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.949'MB/sec zip pipeline throughput = 82.949'MB/sec entries = 929529 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.39 avg-zval = 33.50 usrkeys = 929529 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.39 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929529 bytes = 3485744 } seq expand size = 6506703 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4440 value = 3.4743 dict = 1.00 all = 3.5800 } Zip/UnZip{ index = 0.0957 value = 0.2878 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 11.128501 GB avg = 0.157 KB (by entry num) total key len = 1.965455 GB avg = 0.028 KB total ukey len = 1.372616 GB avg = 0.016 KB total ukey num = 0.084287351 Billion total entry num = 0.071095080 Billion write speed all = 0.559559901 MB/sec (with seq num) write speed all = 0.535254361 MB/sec (without seq num) 2024/11/23-15:25:12.713022 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266355: 929529 keys, 37491512 bytes, temperature: kWarm 2024/11/23-15:25:12.713092 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346712713056, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266355, "file_size": 37491512, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 31140448, "index_size": 1780032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26026812, "raw_average_key_size": 28, "raw_value_size": 108191022, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929529, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346709, "oldest_key_time": 0, "file_creation_time": 1732346709, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266355, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:25:12.747671 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266356.sst) /dev/shm/ToplingTemp/Topling-93175-uezwXG 2024/11/23-15:25:13.752544 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266356.sst): first pass time = 1.00's, 133.572'MB/sec 2024/11/23-15:25:15.148669 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266356.sst): old prealloc_size = 147639500, real_size = 35088352 2024/11/23-15:25:15.149221 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266356.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929007 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.47 avg-zval = 35.60 2024/11/23-15:25:15.149679 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266356.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:25:15.924617 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266356.sst): second pass time = 1.40's, 77.549'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 48.477'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3360.215'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.357'MB/sec write SST data time = 0.13's, 300.492'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23409.224'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.549'MB/sec zip pipeline throughput = 77.549'MB/sec entries = 929007 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.47 avg-zval = 35.60 usrkeys = 929007 avg-key = 20.00 avg-zkey = 1.92 avg-val = 119.75 avg-zval = 35.60 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 435287 bytes = 1523520 } seq expand size = 3047009 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4426 value = 3.2714 dict = 1.00 all = 3.5682 } Zip/UnZip{ index = 0.0958 value = 0.3057 dict = 1.00 all = 0.2803 } ---------------------------- total value len = 11.239753 GB avg = 0.156 KB (by entry num) total key len = 1.991467 GB avg = 0.028 KB total ukey len = 1.391196 GB avg = 0.016 KB total ukey num = 0.085216358 Billion total entry num = 0.072024087 Billion write speed all = 0.565347125 MB/sec (with seq num) write speed all = 0.540727409 MB/sec (without seq num) 2024/11/23-15:25:15.970357 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266356: 929007 keys, 37614921 bytes, temperature: kWarm 2024/11/23-15:25:15.970426 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346715970391, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266356, "file_size": 37614921, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 33076624, "index_size": 1779264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26012196, "raw_average_key_size": 28, "raw_value_size": 108205610, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929007, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346712, "oldest_key_time": 0, "file_creation_time": 1732346712, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266356, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:25:16.011104 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266357.sst) /dev/shm/ToplingTemp/Topling-93175-FaVxLD 2024/11/23-15:25:17.014565 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266357.sst): first pass time = 1.00's, 133.761'MB/sec 2024/11/23-15:25:18.503677 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266357.sst): old prealloc_size = 147639500, real_size = 36913856 2024/11/23-15:25:18.504248 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266357.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926807 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.82 avg-zval = 37.65 2024/11/23-15:25:18.504475 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266357.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:25:19.213041 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266357.sst): second pass time = 1.49's, 72.745'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 54.135'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3259.803'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.394'MB/sec write SST data time = 0.12's, 316.402'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35646.702'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.745'MB/sec zip pipeline throughput = 72.745'MB/sec entries = 926807 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.82 avg-zval = 37.65 usrkeys = 926807 avg-key = 20.00 avg-zkey = 1.93 avg-val = 117.98 avg-zval = 37.65 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.3787 value = 3.1026 dict = 1.00 all = 3.4953 } Zip/UnZip{ index = 0.0964 value = 0.3223 dict = 1.00 all = 0.2861 } ---------------------------- total value len = 11.349101 GB avg = 0.156 KB (by entry num) total key len = 2.017418 GB avg = 0.028 KB total ukey len = 1.409733 GB avg = 0.016 KB total ukey num = 0.086143165 Billion total entry num = 0.072950894 Billion write speed all = 0.571047965 MB/sec (with seq num) write speed all = 0.546114945 MB/sec (without seq num) 2024/11/23-15:25:19.259371 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266357: 926807 keys, 38399321 bytes, temperature: kWarm 2024/11/23-15:25:19.259441 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346719259404, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266357, "file_size": 38399321, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34895984, "index_size": 1785984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950596, "raw_average_key_size": 28, "raw_value_size": 108267262, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926807, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346715, "oldest_key_time": 0, "file_creation_time": 1732346715, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266357, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:25:19.305056 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266358.sst) /dev/shm/ToplingTemp/Topling-93175-np7rZC 2024/11/23-15:25:20.349258 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266358.sst): first pass time = 1.04's, 128.541'MB/sec 2024/11/23-15:25:21.657439 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266358.sst): old prealloc_size = 147639500, real_size = 35786672 2024/11/23-15:25:21.657972 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266358.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923050 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1084 GB zip-val = 0.0338 GB avg-val = 117.41 avg-zval = 36.63 2024/11/23-15:25:21.658702 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266358.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:25:22.484674 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266358.sst): second pass time = 1.31's, 82.890'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.965'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3273.795'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 306.091'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30679.004'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.890'MB/sec zip pipeline throughput = 82.890'MB/sec entries = 923050 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.41 avg-zval = 36.63 usrkeys = 923050 avg-key = 20.00 avg-zkey = 1.89 avg-val = 124.14 avg-zval = 36.63 TagRS{ kinds = RS_Key0_TagN bytes = 144320 } TagArray{ size = 887586 bytes = 2884672 } seq expand size = 6213102 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0338 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5827 value = 3.2052 dict = 1.00 all = 3.3828 } Zip/UnZip{ index = 0.0945 value = 0.3120 dict = 1.00 all = 0.2956 } ---------------------------- total value len = 11.463687 GB avg = 0.155 KB (by entry num) total key len = 2.043263 GB avg = 0.028 KB total ukey len = 1.428194 GB avg = 0.016 KB total ukey num = 0.087066215 Billion total entry num = 0.073873944 Billion write speed all = 0.576966846 MB/sec (with seq num) write speed all = 0.551721877 MB/sec (without seq num) 2024/11/23-15:25:22.532645 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266358: 923050 keys, 39677082 bytes, temperature: kWarm 2024/11/23-15:25:22.532715 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346722532680, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266358, "file_size": 39677082, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4065295908, "table_properties": {"data_size": 33811296, "index_size": 1744448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25845400, "raw_average_key_size": 28, "raw_value_size": 108372368, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923050, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346719, "oldest_key_time": 0, "file_creation_time": 1732346719, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266358, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:25:22.572574 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266359.sst) /dev/shm/ToplingTemp/Topling-93175-WtATKE 2024/11/23-15:25:22.637989 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266359.sst): first pass time = 0.07's, 237.527'MB/sec 2024/11/23-15:25:22.773243 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266359.sst): old prealloc_size = 147639500, real_size = 4675184 2024/11/23-15:25:22.773359 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266359.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 107408 prefix = 4 raw-key = 0.0017 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0125 GB zip-val = 0.0044 GB avg-val = 116.55 avg-zval = 41.36 2024/11/23-15:25:22.773480 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266359.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.016 GB 2024/11/23-15:25:22.865866 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00/266359.sst): second pass time = 0.13's, 92.994'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.09's, 55.243'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2172.274'MB/sec (index lex order gen) rebuild zvType time = 0.08's, 0.346'MB/sec write SST data time = 0.01's, 342.924'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.130'MB, throughput = 4283.786'MB/sec zip my value time = 0.13's, unzip length = 0.013'GB zip my value throughput = 92.994'MB/sec zip pipeline throughput = 92.994'MB/sec entries = 107408 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.55 avg-zval = 41.36 usrkeys = 107408 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.55 avg-zval = 41.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 107408 bytes = 335664 } seq expand size = 751856 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0021 GB value = 0.0125 GB dict = 0.13 MB all = 0.0155 GB } __ZipSize{ index = 0.0002 GB value = 0.0044 GB dict = 0.13 MB all = 0.0051 GB } UnZip/Zip{ index = 10.4401 value = 2.8179 dict = 1.00 all = 3.0348 } Zip/UnZip{ index = 0.0958 value = 0.3549 dict = 1.00 all = 0.3295 } ---------------------------- total value len = 11.476957 GB avg = 0.155 KB (by entry num) total key len = 2.046270 GB avg = 0.028 KB total ukey len = 1.430342 GB avg = 0.016 KB total ukey num = 0.087173623 Billion total entry num = 0.073981352 Billion write speed all = 0.577652748 MB/sec (with seq num) write speed all = 0.552371486 MB/sec (without seq num) 2024/11/23-15:25:22.877985 139957618013760 [compaction_job.cc:1948] [default] [JOB 29193] Generated table #266359: 107408 keys, 5115893 bytes, temperature: kWarm 2024/11/23-15:25:22.878035 139957618013760 EVENT_LOG_v1 {"time_micros": 1732346722878008, "cf_name": "default", "job": 29193, "event": "table_file_creation", "file_number": 266359, "file_size": 5115893, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4043687218, "largest_seqno": 4063775773, "table_properties": {"data_size": 4442400, "index_size": 205760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3007424, "raw_average_key_size": 28, "raw_value_size": 12518076, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 107408, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346722, "oldest_key_time": 0, "file_creation_time": 1732346722, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 266359, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:25:22.890262 139957618013760 [compaction_job.cc:703] job-29193: subcompact[0], size: 0.870978 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29193/att-00 [266336,266337,266338,266339,266340,266341,266342,266343,266344,266345,266346,266347,266348,266349,266350,266351,266352,266353,266354,266355,266356,266357,266358,266359] 2024/11/23-15:25:23.036202 139957618013760 (Original Log Time 2024/11/23-15:23:59.056567) EVENT_LOG_v1 {"time_micros": 1732346639056538, "job": 29193, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [266322], "files_L3": [266274, 266275, 266276, 266277, 266278, 266279, 266280, 266281, 266282, 266283, 266284, 266285, 266286, 266287, 266288, 266289, 266290, 266291, 266292, 266293, 266294, 266295, 266299], "score": 0.999709, "input_data_size": 889054596, "oldest_snapshot_seqno": 4065345444} 2024/11/23-15:25:23.036206 139957618013760 (Original Log Time 2024/11/23-15:25:23.034578) [compaction_job.cc:2022] [default] [JOB 29193] Compacted 1@2 + 23@3 files to L3 => 870977799 bytes 2024/11/23-15:25:23.036207 139957618013760 (Original Log Time 2024/11/23-15:25:23.036126) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 24 0 0 0] max score 0.03, MB/sec: 10.6 rd, 10.4 wr, level 3, files in(1, 23) out(24 +0 blob) MB in(24.3, 823.6 +0.0 blob) out(830.6 +0.0 blob), read-write-amplify(69.1) write-amplify(34.2) OK, records in: 21725154, records dropped: 101966 output_compression: Snappy 2024/11/23-15:25:23.036210 139957618013760 (Original Log Time 2024/11/23-15:25:23.036176) EVENT_LOG_v1 {"time_micros": 1732346723036156, "cf": "default", "job": 29193, "event": "compaction_finished", "compaction_time_micros": 83833701, "compaction_time_cpu_micros": 35038926, "output_level": 3, "num_output_files": 24, "total_output_size": 870977799, "num_input_records": 21725154, "num_output_records": 21623188, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 647400140, "file_range_sync_nanos": 0, "file_fsync_nanos": 17431, "file_prepare_write_nanos": 65198, "lsm_state": [0, 0, 0, 24, 0, 0, 0]} 2024/11/23-15:25:23.165538 139957618013760 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732346638,"executesMs":84116,"compactionJobId":29193,"attempt":0,"compactionInputRawBytes":3117013949,"compactionInputZipBytes":889054596,"compactionOutputRawBytes":0,"compactionOutputZipBytes":870977799,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5f05e07f4dd7f05ff0221d29e49699d4-daf873c2231615fb-00"}}, response = 2024/11/23-15:25:23.165634 139957618013760 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29193/att-00: olev 3, work 84.116 s, result 7.773 ms, install 1.647 ms, input{raw 2.903 GiB zip 847.869 MiB}