2024/11/23-18:28:13.848292 139957592835648 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00: fork to child time = 0.034020 sec 2024/11/23-18:28:13.849316 139957592835648 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00 2024/11/23-18:28:13.857673 139957592835648 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00 2024/11/23-18:28:13.858897 139957592835648 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-18:28:13.859289 139957592835648 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-18:28:13.921967 139957592835648 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33103/att-00/MANIFEST-206347 2024/11/23-18:28:13.955991 139957592835648 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33103/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 274846, last_sequence is 4183315014, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 274681 2024/11/23-18:28:13.956004 139957592835648 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-18:28:13.956209 139957592835648 [version_set.cc:5721] Creating manifest 274846 2024/11/23-18:28:13.992562 139957592835648 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-18:28:14.019589 139957592835648 [compaction_job.cc:2437] [default] [JOB 33103] Compacting 1@2 + 19@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-18:28:14.019621 139957592835648 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24791 Base level 2, inputs: [274816(25MB)], [274766(36MB) 274767(36MB) 274768(36MB) 274769(37MB) 274770(38MB) 274771(37MB) 274772(37MB) 274773(35MB) 274774(37MB) 274775(34MB) 274776(35MB) 274778(37MB) 274780(35MB) 274782(37MB) 274783(35MB) 274785(35MB) 274786(36MB) 274787(37MB) 274839(36MB)] 2024/11/23-18:28:14.027760 139957592835648 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:28:14.027802 139957592835648 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:28:14.027832 139957592835648 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:28:14.027841 139957592835648 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:28:14.027859 139957592835648 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:28:14.629298 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274847.sst) /dev/shm/ToplingTemp/Topling-150017-sDGy3L 2024/11/23-18:28:15.198035 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274847.sst): first pass time = 0.57's, 236.011'MB/sec 2024/11/23-18:28:17.058551 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274847.sst): old prealloc_size = 147639500, real_size = 34657456 2024/11/23-18:28:17.059141 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274847.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925274 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 117.06 avg-zval = 35.30 2024/11/23-18:28:17.060030 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274847.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:18.098181 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274847.sst): second pass time = 1.86's, 58.257'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 37.052'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3097.997'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.254'MB/sec write SST data time = 0.13's, 299.632'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32651.118'MB/sec zip my value time = 1.86's, unzip length = 0.108'GB zip my value throughput = 58.257'MB/sec zip pipeline throughput = 84.617'MB/sec entries = 925274 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.06 avg-zval = 35.30 usrkeys = 925274 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.06 avg-zval = 35.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925274 bytes = 3007152 } seq expand size = 6476918 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5133 value = 3.3157 dict = 1.00 all = 3.4844 } Zip/UnZip{ index = 0.0951 value = 0.3016 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 8.963407 GB avg = 0.168 KB (by entry num) total key len = 1.466729 GB avg = 0.028 KB total ukey len = 1.016383 GB avg = 0.015 KB total ukey num = 0.066475709 Billion total entry num = 0.053283438 Billion write speed all = 0.303326091 MB/sec (with seq num) write speed all = 0.290929507 MB/sec (without seq num) 2024/11/23-18:28:18.161595 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274847: 925274 keys, 38519872 bytes, temperature: kWarm 2024/11/23-18:28:18.161693 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357698161633, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274847, "file_size": 38519872, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4136224546, "largest_seqno": 4183069319, "table_properties": {"data_size": 32665760, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25907672, "raw_average_key_size": 28, "raw_value_size": 108310180, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925274, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357694, "oldest_key_time": 0, "file_creation_time": 1732357694, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274847, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:18.201340 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274848.sst) /dev/shm/ToplingTemp/Topling-150017-CG7u4M 2024/11/23-18:28:19.198543 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274848.sst): first pass time = 1.00's, 134.599'MB/sec 2024/11/23-18:28:20.962039 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274848.sst): old prealloc_size = 147639500, real_size = 34053616 2024/11/23-18:28:20.962736 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274848.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925464 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0320 GB avg-val = 117.03 avg-zval = 34.63 2024/11/23-18:28:20.963689 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274848.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:22.038469 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274848.sst): second pass time = 1.76's, 61.445'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 35.333'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2621.615'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.265'MB/sec write SST data time = 0.20's, 188.908'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35395.696'MB/sec zip my value time = 1.76's, unzip length = 0.108'GB zip my value throughput = 61.445'MB/sec zip pipeline throughput = 77.783'MB/sec entries = 925464 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.03 avg-zval = 34.63 usrkeys = 925464 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.03 avg-zval = 34.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925464 bytes = 3123456 } seq expand size = 6478248 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.4437 value = 3.3793 dict = 1.00 all = 3.5291 } Zip/UnZip{ index = 0.0958 value = 0.2959 dict = 1.00 all = 0.2834 } ---------------------------- total value len = 9.078190 GB avg = 0.167 KB (by entry num) total key len = 1.492642 GB avg = 0.028 KB total ukey len = 1.034893 GB avg = 0.015 KB total ukey num = 0.067401173 Billion total entry num = 0.054208902 Billion write speed all = 0.307382547 MB/sec (with seq num) write speed all = 0.294772095 MB/sec (without seq num) 2024/11/23-18:28:22.086591 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274848: 925464 keys, 38032234 bytes, temperature: kWarm 2024/11/23-18:28:22.086659 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357702086624, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274848, "file_size": 38032234, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4094024708, "largest_seqno": 4183069633, "table_properties": {"data_size": 32049760, "index_size": 1772288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912992, "raw_average_key_size": 28, "raw_value_size": 108304756, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925464, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357698, "oldest_key_time": 0, "file_creation_time": 1732357698, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274848, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:22.126657 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274849.sst) /dev/shm/ToplingTemp/Topling-150017-AFI5BN 2024/11/23-18:28:23.230430 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274849.sst): first pass time = 1.10's, 121.604'MB/sec 2024/11/23-18:28:25.039770 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274849.sst): old prealloc_size = 147639500, real_size = 34301776 2024/11/23-18:28:25.040498 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274849.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928191 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.60 avg-zval = 34.80 2024/11/23-18:28:25.041340 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274849.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:26.008400 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274849.sst): second pass time = 1.81's, 59.862'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 39.528'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2487.112'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.15's, 251.455'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26841.174'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.862'MB/sec zip pipeline throughput = 105.946'MB/sec entries = 928191 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.60 avg-zval = 34.80 usrkeys = 928191 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.60 avg-zval = 34.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928191 bytes = 3132656 } seq expand size = 6497337 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5147 value = 3.3503 dict = 1.00 all = 3.5055 } Zip/UnZip{ index = 0.0951 value = 0.2985 dict = 1.00 all = 0.2853 } ---------------------------- total value len = 9.192916 GB avg = 0.167 KB (by entry num) total key len = 1.518631 GB avg = 0.028 KB total ukey len = 1.053457 GB avg = 0.015 KB total ukey num = 0.068329364 Billion total entry num = 0.055137093 Billion write speed all = 0.311438362 MB/sec (with seq num) write speed all = 0.298613468 MB/sec (without seq num) 2024/11/23-18:28:26.097070 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274849: 928191 keys, 38287760 bytes, temperature: kWarm 2024/11/23-18:28:26.097171 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357706097088, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274849, "file_size": 38287760, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668904, "largest_seqno": 4183069431, "table_properties": {"data_size": 32304064, "index_size": 1765504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989348, "raw_average_key_size": 28, "raw_value_size": 108228470, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928191, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357702, "oldest_key_time": 0, "file_creation_time": 1732357702, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274849, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:26.155034 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274850.sst) /dev/shm/ToplingTemp/Topling-150017-2es4gK 2024/11/23-18:28:27.442191 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274850.sst): first pass time = 1.29's, 104.278'MB/sec 2024/11/23-18:28:28.941841 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274850.sst): old prealloc_size = 147639500, real_size = 34646576 2024/11/23-18:28:28.942468 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274850.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927092 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 116.77 avg-zval = 35.21 2024/11/23-18:28:28.943212 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274850.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:29.785989 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274850.sst): second pass time = 1.50's, 72.229'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.627'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2935.149'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 306.737'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31401.122'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.229'MB/sec zip pipeline throughput = 72.229'MB/sec entries = 927092 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.77 avg-zval = 35.21 usrkeys = 927092 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.77 avg-zval = 35.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927092 bytes = 3013056 } seq expand size = 6489644 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4459 value = 3.3168 dict = 1.00 all = 3.4849 } Zip/UnZip{ index = 0.0957 value = 0.3015 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 9.307665 GB avg = 0.166 KB (by entry num) total key len = 1.544590 GB avg = 0.028 KB total ukey len = 1.071998 GB avg = 0.015 KB total ukey num = 0.069256456 Billion total entry num = 0.056064185 Billion write speed all = 0.315494781 MB/sec (with seq num) write speed all = 0.302455677 MB/sec (without seq num) 2024/11/23-18:28:29.835175 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274850: 927092 keys, 38514288 bytes, temperature: kWarm 2024/11/23-18:28:29.835251 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357709835210, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274850, "file_size": 38514288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4131604997, "largest_seqno": 4183069265, "table_properties": {"data_size": 32639584, "index_size": 1775040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958576, "raw_average_key_size": 28, "raw_value_size": 108259208, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927092, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357706, "oldest_key_time": 0, "file_creation_time": 1732357706, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274850, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:29.878299 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274851.sst) /dev/shm/ToplingTemp/Topling-150017-Th6azK 2024/11/23-18:28:30.949189 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274851.sst): first pass time = 1.07's, 125.338'MB/sec 2024/11/23-18:28:32.815275 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274851.sst): old prealloc_size = 147639500, real_size = 36700144 2024/11/23-18:28:32.815807 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274851.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934732 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0347 GB avg-val = 115.59 avg-zval = 37.10 2024/11/23-18:28:32.816586 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274851.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:33.609027 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274851.sst): second pass time = 1.87's, 57.923'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 51.237'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3505.156'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.352'MB/sec write SST data time = 0.13's, 313.896'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33447.251'MB/sec zip my value time = 1.87's, unzip length = 0.108'GB zip my value throughput = 57.923'MB/sec zip pipeline throughput = 128.490'MB/sec entries = 934732 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.59 avg-zval = 37.10 usrkeys = 934732 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.51 avg-zval = 37.10 TagRS{ kinds = RS_Key0_TagN bytes = 146160 } TagArray{ size = 791201 bytes = 2967024 } seq expand size = 5538407 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0407 GB } UnZip/Zip{ index = 10.4450 value = 3.1158 dict = 1.00 all = 3.3004 } Zip/UnZip{ index = 0.0957 value = 0.3209 dict = 1.00 all = 0.3030 } ---------------------------- total value len = 9.421248 GB avg = 0.165 KB (by entry num) total key len = 1.570762 GB avg = 0.028 KB total ukey len = 1.090693 GB avg = 0.016 KB total ukey num = 0.070191188 Billion total entry num = 0.056998917 Billion write speed all = 0.319522236 MB/sec (with seq num) write speed all = 0.306267211 MB/sec (without seq num) 2024/11/23-18:28:33.672045 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274851: 934732 keys, 40667266 bytes, temperature: kWarm 2024/11/23-18:28:33.672134 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357713672086, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274851, "file_size": 40667266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4183066383, "table_properties": {"data_size": 34676448, "index_size": 1789824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26172496, "raw_average_key_size": 28, "raw_value_size": 108045332, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934732, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357709, "oldest_key_time": 0, "file_creation_time": 1732357709, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274851, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:33.755179 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274852.sst) /dev/shm/ToplingTemp/Topling-150017-cvkDYM 2024/11/23-18:28:34.824637 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274852.sst): first pass time = 1.07's, 125.508'MB/sec 2024/11/23-18:28:36.161416 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274852.sst): old prealloc_size = 147639500, real_size = 35723696 2024/11/23-18:28:36.162016 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274852.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925240 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0337 GB avg-val = 117.06 avg-zval = 36.45 2024/11/23-18:28:36.162780 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274852.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:36.977588 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274852.sst): second pass time = 1.34's, 81.071'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.502'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3085.406'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 305.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34871.616'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.071'MB/sec zip pipeline throughput = 81.071'MB/sec entries = 925240 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.06 avg-zval = 36.45 usrkeys = 925240 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.83 avg-zval = 36.45 TagRS{ kinds = RS_Key0_TagN bytes = 144680 } TagArray{ size = 761979 bytes = 2857440 } seq expand size = 5333853 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4442 value = 3.2120 dict = 1.00 all = 3.3906 } Zip/UnZip{ index = 0.0957 value = 0.3113 dict = 1.00 all = 0.2949 } ---------------------------- total value len = 9.534893 GB avg = 0.165 KB (by entry num) total key len = 1.596669 GB avg = 0.028 KB total ukey len = 1.109198 GB avg = 0.016 KB total ukey num = 0.071116428 Billion total entry num = 0.057924157 Billion write speed all = 0.323547123 MB/sec (with seq num) write speed all = 0.310078254 MB/sec (without seq num) 2024/11/23-18:28:37.025662 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274852: 925240 keys, 39584856 bytes, temperature: kWarm 2024/11/23-18:28:37.025737 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357717025695, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274852, "file_size": 39584856, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012772426, "table_properties": {"data_size": 33720416, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906720, "raw_average_key_size": 28, "raw_value_size": 108311036, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357713, "oldest_key_time": 0, "file_creation_time": 1732357713, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274852, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:37.066891 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274853.sst) /dev/shm/ToplingTemp/Topling-150017-lBAdRM 2024/11/23-18:28:38.162044 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274853.sst): first pass time = 1.10's, 122.562'MB/sec 2024/11/23-18:28:40.020069 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274853.sst): old prealloc_size = 147639500, real_size = 35022288 2024/11/23-18:28:40.020587 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274853.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928829 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0330 GB avg-val = 116.50 avg-zval = 35.54 2024/11/23-18:28:40.022273 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274853.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:40.861301 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274853.sst): second pass time = 1.86's, 58.306'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.650'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3543.435'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 303.367'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33718.676'MB/sec zip my value time = 1.86's, unzip length = 0.108'GB zip my value throughput = 58.306'MB/sec zip pipeline throughput = 122.385'MB/sec entries = 928829 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 35.54 usrkeys = 928829 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 35.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928829 bytes = 3367024 } seq expand size = 6501803 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4451 value = 3.2780 dict = 1.00 all = 3.4202 } Zip/UnZip{ index = 0.0957 value = 0.3051 dict = 1.00 all = 0.2924 } ---------------------------- total value len = 9.649606 GB avg = 0.164 KB (by entry num) total key len = 1.622676 GB avg = 0.028 KB total ukey len = 1.127774 GB avg = 0.016 KB total ukey num = 0.072045257 Billion total entry num = 0.058852986 Billion write speed all = 0.327600263 MB/sec (with seq num) write speed all = 0.313916962 MB/sec (without seq num) 2024/11/23-18:28:40.909748 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274853: 928829 keys, 39242504 bytes, temperature: kWarm 2024/11/23-18:28:40.909819 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357720909783, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274853, "file_size": 39242504, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33011392, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007212, "raw_average_key_size": 28, "raw_value_size": 108210614, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357717, "oldest_key_time": 0, "file_creation_time": 1732357717, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274853, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:40.947248 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274854.sst) /dev/shm/ToplingTemp/Topling-150017-KPA1XL 2024/11/23-18:28:41.918692 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274854.sst): first pass time = 0.97's, 138.168'MB/sec 2024/11/23-18:28:43.244071 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274854.sst): old prealloc_size = 147639500, real_size = 33055712 2024/11/23-18:28:43.244651 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274854.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928305 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.58 avg-zval = 33.46 2024/11/23-18:28:43.246824 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274854.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:44.090440 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274854.sst): second pass time = 1.32's, 81.704'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.042'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3163.865'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 287.816'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28354.937'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.704'MB/sec zip pipeline throughput = 81.704'MB/sec entries = 928305 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.58 avg-zval = 33.46 usrkeys = 928305 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.58 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928305 bytes = 3365120 } seq expand size = 6498135 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5149 value = 3.4846 dict = 1.00 all = 3.6008 } Zip/UnZip{ index = 0.0951 value = 0.2870 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 9.764329 GB avg = 0.163 KB (by entry num) total key len = 1.648669 GB avg = 0.028 KB total ukey len = 1.146341 GB avg = 0.016 KB total ukey num = 0.072973562 Billion total entry num = 0.059781291 Billion write speed all = 0.331658688 MB/sec (with seq num) write speed all = 0.317760860 MB/sec (without seq num) 2024/11/23-18:28:44.136115 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274854: 928305 keys, 37274332 bytes, temperature: kWarm 2024/11/23-18:28:44.136188 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357724136151, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274854, "file_size": 37274332, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012776021, "table_properties": {"data_size": 31057744, "index_size": 1765696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25992540, "raw_average_key_size": 28, "raw_value_size": 108225198, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928305, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357720, "oldest_key_time": 0, "file_creation_time": 1732357720, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274854, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:44.182274 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274855.sst) /dev/shm/ToplingTemp/Topling-150017-vzPJoN 2024/11/23-18:28:45.222541 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274855.sst): first pass time = 1.04's, 129.027'MB/sec 2024/11/23-18:28:46.558631 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274855.sst): old prealloc_size = 147639500, real_size = 35703792 2024/11/23-18:28:46.559171 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274855.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923768 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0337 GB avg-val = 117.29 avg-zval = 36.49 2024/11/23-18:28:46.559879 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274855.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:47.367325 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274855.sst): second pass time = 1.34's, 81.146'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 48.829'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3423.681'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 306.959'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31693.474'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.146'MB/sec zip pipeline throughput = 81.146'MB/sec entries = 923768 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.29 avg-zval = 36.49 usrkeys = 923768 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.91 avg-zval = 36.49 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 740771 bytes = 2777904 } seq expand size = 5185397 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4446 value = 3.2148 dict = 1.00 all = 3.3990 } Zip/UnZip{ index = 0.0957 value = 0.3111 dict = 1.00 all = 0.2942 } ---------------------------- total value len = 9.877867 GB avg = 0.163 KB (by entry num) total key len = 1.674534 GB avg = 0.028 KB total ukey len = 1.164816 GB avg = 0.016 KB total ukey num = 0.073897330 Billion total entry num = 0.060705059 Billion write speed all = 0.335677742 MB/sec (with seq num) write speed all = 0.321566503 MB/sec (without seq num) 2024/11/23-18:28:47.414911 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274855: 923768 keys, 39487306 bytes, temperature: kWarm 2024/11/23-18:28:47.414990 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357727414945, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274855, "file_size": 39487306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 33703776, "index_size": 1768896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25865504, "raw_average_key_size": 28, "raw_value_size": 108352288, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923768, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357724, "oldest_key_time": 0, "file_creation_time": 1732357724, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274855, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:47.453487 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274856.sst) /dev/shm/ToplingTemp/Topling-150017-WCGhpL 2024/11/23-18:28:48.460656 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274856.sst): first pass time = 1.01's, 133.267'MB/sec 2024/11/23-18:28:49.789092 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274856.sst): old prealloc_size = 147639500, real_size = 32444176 2024/11/23-18:28:49.789681 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274856.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.79 avg-zval = 32.60 2024/11/23-18:28:49.790499 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274856.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:50.651859 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274856.sst): second pass time = 1.33's, 81.408'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.513'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3133.171'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 288.134'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32455.345'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.408'MB/sec zip pipeline throughput = 81.408'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 32.60 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 32.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5153 value = 3.5512 dict = 1.00 all = 3.6593 } Zip/UnZip{ index = 0.0951 value = 0.2816 dict = 1.00 all = 0.2733 } ---------------------------- total value len = 9.992482 GB avg = 0.162 KB (by entry num) total key len = 1.700671 GB avg = 0.028 KB total ukey len = 1.183485 GB avg = 0.016 KB total ukey num = 0.074830787 Billion total entry num = 0.061638516 Billion write speed all = 0.339735146 MB/sec (with seq num) write speed all = 0.325408286 MB/sec (without seq num) 2024/11/23-18:28:50.696602 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274856: 933457 keys, 36678306 bytes, temperature: kWarm 2024/11/23-18:28:50.696671 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357730696637, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274856, "file_size": 36678306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 30435200, "index_size": 1775424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357727, "oldest_key_time": 0, "file_creation_time": 1732357727, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274856, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:50.745772 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274857.sst) /dev/shm/ToplingTemp/Topling-150017-WaiOwN 2024/11/23-18:28:51.775334 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274857.sst): first pass time = 1.03's, 130.370'MB/sec 2024/11/23-18:28:53.094247 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274857.sst): old prealloc_size = 147639500, real_size = 32520976 2024/11/23-18:28:53.094797 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274857.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935111 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0305 GB avg-val = 115.53 avg-zval = 32.60 2024/11/23-18:28:53.096932 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274857.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:53.950205 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274857.sst): second pass time = 1.32's, 81.966'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.948'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3418.041'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.322'MB/sec write SST data time = 0.13's, 282.802'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30238.205'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.966'MB/sec zip pipeline throughput = 81.966'MB/sec entries = 935111 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.53 avg-zval = 32.60 usrkeys = 935111 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.53 avg-zval = 32.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935111 bytes = 3389792 } seq expand size = 6545777 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.3776 value = 3.5439 dict = 1.00 all = 3.6511 } Zip/UnZip{ index = 0.0964 value = 0.2822 dict = 1.00 all = 0.2739 } ---------------------------- total value len = 10.107062 GB avg = 0.162 KB (by entry num) total key len = 1.726854 GB avg = 0.028 KB total ukey len = 1.202187 GB avg = 0.016 KB total ukey num = 0.075765898 Billion total entry num = 0.062573627 Billion write speed all = 0.343791975 MB/sec (with seq num) write speed all = 0.329249158 MB/sec (without seq num) 2024/11/23-18:28:53.995081 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274857: 935111 keys, 36760696 bytes, temperature: kWarm 2024/11/23-18:28:53.995154 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357733995117, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274857, "file_size": 36760696, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30484864, "index_size": 1802176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26183108, "raw_average_key_size": 28, "raw_value_size": 108034762, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935111, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357730, "oldest_key_time": 0, "file_creation_time": 1732357730, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274857, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:54.030878 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274858.sst) /dev/shm/ToplingTemp/Topling-150017-2FKenL 2024/11/23-18:28:55.077712 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274858.sst): first pass time = 1.05's, 128.219'MB/sec 2024/11/23-18:28:56.424704 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274858.sst): old prealloc_size = 147639500, real_size = 35017168 2024/11/23-18:28:56.425305 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274858.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0330 GB avg-val = 117.01 avg-zval = 35.67 2024/11/23-18:28:56.426150 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274858.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:28:57.265853 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274858.sst): second pass time = 1.35's, 80.482'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.774'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3060.929'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 313.893'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31130.402'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.482'MB/sec zip pipeline throughput = 80.482'MB/sec entries = 925568 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.01 avg-zval = 35.67 usrkeys = 925568 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.01 avg-zval = 35.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925568 bytes = 3470896 } seq expand size = 6478976 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4434 value = 3.2806 dict = 1.00 all = 3.4115 } Zip/UnZip{ index = 0.0958 value = 0.3048 dict = 1.00 all = 0.2931 } ---------------------------- total value len = 10.221843 GB avg = 0.161 KB (by entry num) total key len = 1.752770 GB avg = 0.028 KB total ukey len = 1.220699 GB avg = 0.016 KB total ukey num = 0.076691466 Billion total entry num = 0.063499195 Billion write speed all = 0.347845907 MB/sec (with seq num) write speed all = 0.333089398 MB/sec (without seq num) 2024/11/23-18:28:57.314009 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274858: 925568 keys, 39342954 bytes, temperature: kWarm 2024/11/23-18:28:57.314080 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357737314045, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274858, "file_size": 39342954, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33013056, "index_size": 1772544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915904, "raw_average_key_size": 28, "raw_value_size": 108301824, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357734, "oldest_key_time": 0, "file_creation_time": 1732357734, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274858, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:28:57.353377 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274859.sst) /dev/shm/ToplingTemp/Topling-150017-hcDcIK 2024/11/23-18:28:58.374776 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274859.sst): first pass time = 1.02's, 131.412'MB/sec 2024/11/23-18:28:59.724903 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274859.sst): old prealloc_size = 147639500, real_size = 32406288 2024/11/23-18:28:59.725507 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.13 avg-zval = 32.63 2024/11/23-18:28:59.726335 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274859.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:29:00.568151 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274859.sst): second pass time = 1.35's, 80.151'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.445'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3074.046'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 290.975'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32825.521'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.151'MB/sec zip pipeline throughput = 80.151'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.13 avg-zval = 32.63 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.13 avg-zval = 32.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4426 value = 3.5585 dict = 1.00 all = 3.6638 } Zip/UnZip{ index = 0.0958 value = 0.2810 dict = 1.00 all = 0.2729 } ---------------------------- total value len = 10.336505 GB avg = 0.160 KB (by entry num) total key len = 1.778845 GB avg = 0.028 KB total ukey len = 1.239323 GB avg = 0.016 KB total ukey num = 0.077622708 Billion total entry num = 0.064430437 Billion write speed all = 0.351900349 MB/sec (with seq num) write speed all = 0.336928866 MB/sec (without seq num) 2024/11/23-18:29:00.612624 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274859: 931242 keys, 36633280 bytes, temperature: kWarm 2024/11/23-18:29:00.612697 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357740612661, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274859, "file_size": 36633280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 30389760, "index_size": 1783552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357737, "oldest_key_time": 0, "file_creation_time": 1732357737, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:29:00.648395 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274860.sst) /dev/shm/ToplingTemp/Topling-150017-kZBSaN 2024/11/23-18:29:01.695217 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274860.sst): first pass time = 1.05's, 128.220'MB/sec 2024/11/23-18:29:03.035855 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274860.sst): old prealloc_size = 147639500, real_size = 35078160 2024/11/23-18:29:03.036414 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0331 GB avg-val = 117.46 avg-zval = 35.86 2024/11/23-18:29:03.038609 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274860.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:29:03.880569 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274860.sst): second pass time = 1.34's, 80.887'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.640'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3228.698'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 311.344'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31612.540'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.887'MB/sec zip pipeline throughput = 80.887'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 35.86 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 35.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.5146 value = 3.2752 dict = 1.00 all = 3.4069 } Zip/UnZip{ index = 0.0951 value = 0.3053 dict = 1.00 all = 0.2935 } ---------------------------- total value len = 10.451346 GB avg = 0.160 KB (by entry num) total key len = 1.804680 GB avg = 0.028 KB total ukey len = 1.257777 GB avg = 0.016 KB total ukey num = 0.078545405 Billion total entry num = 0.065353134 Billion write speed all = 0.355952172 MB/sec (with seq num) write speed all = 0.340767746 MB/sec (without seq num) 2024/11/23-18:29:03.927605 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274860: 922697 keys, 39396438 bytes, temperature: kWarm 2024/11/23-18:29:03.927669 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357743927636, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274860, "file_size": 39396438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33092224, "index_size": 1755072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357740, "oldest_key_time": 0, "file_creation_time": 1732357740, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:29:03.965101 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274861.sst) /dev/shm/ToplingTemp/Topling-150017-2MAwoN 2024/11/23-18:29:04.994783 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274861.sst): first pass time = 1.03's, 130.353'MB/sec 2024/11/23-18:29:06.356335 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274861.sst): old prealloc_size = 147639500, real_size = 32857120 2024/11/23-18:29:06.356867 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.15 avg-zval = 33.12 2024/11/23-18:29:06.357740 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274861.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:29:07.224152 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274861.sst): second pass time = 1.36's, 79.480'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.867'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3503.126'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 297.090'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29048.327'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.480'MB/sec zip pipeline throughput = 79.480'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 33.12 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 33.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4430 value = 3.5066 dict = 1.00 all = 3.6080 } Zip/UnZip{ index = 0.0958 value = 0.2852 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 10.566011 GB avg = 0.159 KB (by entry num) total key len = 1.830751 GB avg = 0.028 KB total ukey len = 1.276399 GB avg = 0.016 KB total ukey num = 0.079476488 Billion total entry num = 0.066284217 Billion write speed all = 0.360004596 MB/sec (with seq num) write speed all = 0.344605334 MB/sec (without seq num) 2024/11/23-18:29:07.268939 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274861: 931083 keys, 37199884 bytes, temperature: kWarm 2024/11/23-18:29:07.269025 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357747268971, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274861, "file_size": 37199884, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30840976, "index_size": 1783168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357743, "oldest_key_time": 0, "file_creation_time": 1732357743, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:29:07.307583 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274862.sst) /dev/shm/ToplingTemp/Topling-150017-bx6Z4K 2024/11/23-18:29:08.319347 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274862.sst): first pass time = 1.01's, 132.663'MB/sec 2024/11/23-18:29:09.705655 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274862.sst): old prealloc_size = 147639500, real_size = 34278560 2024/11/23-18:29:09.706215 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927935 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.64 avg-zval = 34.78 2024/11/23-18:29:09.706771 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274862.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:29:10.499869 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274862.sst): second pass time = 1.38's, 78.151'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 46.901'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3313.655'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.348'MB/sec write SST data time = 0.13's, 295.338'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35911.549'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.151'MB/sec zip pipeline throughput = 78.151'MB/sec entries = 927935 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.64 avg-zval = 34.78 usrkeys = 927935 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.88 avg-zval = 34.78 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 561703 bytes = 1965968 } seq expand size = 3931921 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4426 value = 3.3541 dict = 1.00 all = 3.6033 } Zip/UnZip{ index = 0.0958 value = 0.2981 dict = 1.00 all = 0.2775 } ---------------------------- total value len = 10.678179 GB avg = 0.159 KB (by entry num) total key len = 1.856733 GB avg = 0.028 KB total ukey len = 1.294958 GB avg = 0.016 KB total ukey num = 0.080404423 Billion total entry num = 0.067212152 Billion write speed all = 0.363981870 MB/sec (with seq num) write speed all = 0.348368514 MB/sec (without seq num) 2024/11/23-18:29:10.548253 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274862: 927935 keys, 37248112 bytes, temperature: kWarm 2024/11/23-18:29:10.548323 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357750548288, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274862, "file_size": 37248112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 32269200, "index_size": 1777216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982180, "raw_average_key_size": 28, "raw_value_size": 108235650, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927935, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357747, "oldest_key_time": 0, "file_creation_time": 1732357747, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:29:10.585859 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274863.sst) /dev/shm/ToplingTemp/Topling-150017-r7rz3K 2024/11/23-18:29:11.588116 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274863.sst): first pass time = 1.00's, 133.921'MB/sec 2024/11/23-18:29:13.083158 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274863.sst): old prealloc_size = 147639500, real_size = 36815872 2024/11/23-18:29:13.083740 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926695 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.83 avg-zval = 37.58 2024/11/23-18:29:13.083969 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274863.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:29:13.797330 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274863.sst): second pass time = 1.49's, 72.457'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.633'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3126.137'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.392'MB/sec write SST data time = 0.12's, 310.598'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32956.898'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.457'MB/sec zip pipeline throughput = 72.457'MB/sec entries = 926695 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 37.58 usrkeys = 926695 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.00 avg-zval = 37.58 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5153 value = 3.1093 dict = 1.00 all = 3.5042 } Zip/UnZip{ index = 0.0951 value = 0.3216 dict = 1.00 all = 0.2854 } ---------------------------- total value len = 10.787530 GB avg = 0.158 KB (by entry num) total key len = 1.882680 GB avg = 0.028 KB total ukey len = 1.313492 GB avg = 0.016 KB total ukey num = 0.081331118 Billion total entry num = 0.068138847 Billion write speed all = 0.367875373 MB/sec (with seq num) write speed all = 0.352048261 MB/sec (without seq num) 2024/11/23-18:29:13.848696 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274863: 926695 keys, 38301561 bytes, temperature: kWarm 2024/11/23-18:29:13.848768 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357753848728, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274863, "file_size": 38301561, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34821424, "index_size": 1762560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947460, "raw_average_key_size": 28, "raw_value_size": 108270370, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926695, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357750, "oldest_key_time": 0, "file_creation_time": 1732357750, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:29:13.884097 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274864.sst) /dev/shm/ToplingTemp/Topling-150017-G5eKbO 2024/11/23-18:29:14.868637 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274864.sst): first pass time = 0.98's, 136.330'MB/sec 2024/11/23-18:29:16.208440 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274864.sst): old prealloc_size = 147639500, real_size = 34899392 2024/11/23-18:29:16.208965 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927095 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0329 GB avg-val = 116.77 avg-zval = 35.50 2024/11/23-18:29:16.209786 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274864.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:29:17.030233 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274864.sst): second pass time = 1.34's, 80.887'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 46.943'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.163'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 301.762'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34228.137'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.887'MB/sec zip pipeline throughput = 80.887'MB/sec entries = 927095 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 35.50 usrkeys = 927095 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.54 avg-zval = 35.50 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 764031 bytes = 2674128 } seq expand size = 5348217 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5840 value = 3.2890 dict = 1.00 all = 3.4793 } Zip/UnZip{ index = 0.0945 value = 0.3040 dict = 1.00 all = 0.2874 } ---------------------------- total value len = 10.901137 GB avg = 0.158 KB (by entry num) total key len = 1.908639 GB avg = 0.028 KB total ukey len = 1.332034 GB avg = 0.016 KB total ukey num = 0.082258213 Billion total entry num = 0.069065942 Billion write speed all = 0.371892718 MB/sec (with seq num) write speed all = 0.355851769 MB/sec (without seq num) 2024/11/23-18:29:17.079350 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274864: 927095 keys, 38576512 bytes, temperature: kWarm 2024/11/23-18:29:17.079418 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357757079384, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274864, "file_size": 38576512, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4183069074, "table_properties": {"data_size": 32915568, "index_size": 1751872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958660, "raw_average_key_size": 28, "raw_value_size": 108259138, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927095, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357753, "oldest_key_time": 0, "file_creation_time": 1732357753, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:29:17.118021 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274865.sst) /dev/shm/ToplingTemp/Topling-150017-WbPK3K 2024/11/23-18:29:18.151513 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274865.sst): first pass time = 1.03's, 129.874'MB/sec 2024/11/23-18:29:19.464346 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274865.sst): old prealloc_size = 147639500, real_size = 33435248 2024/11/23-18:29:19.464918 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925752 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.98 avg-zval = 33.95 2024/11/23-18:29:19.465704 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274865.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:29:20.320943 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274865.sst): second pass time = 1.31's, 82.537'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.679'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3238.260'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 292.463'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28440.542'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.537'MB/sec zip pipeline throughput = 82.537'MB/sec entries = 925752 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.98 avg-zval = 33.95 usrkeys = 925752 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.98 avg-zval = 33.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925752 bytes = 3124432 } seq expand size = 6480264 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4432 value = 3.4456 dict = 1.00 all = 3.5873 } Zip/UnZip{ index = 0.0958 value = 0.2902 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 11.015914 GB avg = 0.157 KB (by entry num) total key len = 1.934560 GB avg = 0.028 KB total ukey len = 1.350549 GB avg = 0.016 KB total ukey num = 0.083183965 Billion total entry num = 0.069991694 Billion write speed all = 0.375941541 MB/sec (with seq num) write speed all = 0.359687133 MB/sec (without seq num) 2024/11/23-18:29:20.368786 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274865: 925752 keys, 37414410 bytes, temperature: kWarm 2024/11/23-18:29:20.368857 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357760368821, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274865, "file_size": 37414410, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4183069597, "table_properties": {"data_size": 31430688, "index_size": 1772928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25921056, "raw_average_key_size": 28, "raw_value_size": 108296752, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925752, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357757, "oldest_key_time": 0, "file_creation_time": 1732357757, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:29:20.407297 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274866.sst) /dev/shm/ToplingTemp/Topling-150017-BZZrZK 2024/11/23-18:29:20.476834 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274866.sst): first pass time = 0.07's, 236.305'MB/sec 2024/11/23-18:29:20.624709 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274866.sst): old prealloc_size = 147639500, real_size = 4916304 2024/11/23-18:29:20.624834 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274866.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 112826 prefix = 4 raw-key = 0.0018 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0133 GB zip-val = 0.0047 GB avg-val = 117.56 avg-zval = 41.40 2024/11/23-18:29:20.624958 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274866.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.016 GB 2024/11/23-18:29:20.728071 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00/274866.sst): second pass time = 0.15's, 90.114'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.10's, 52.066'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2124.159'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.326'MB/sec write SST data time = 0.02's, 320.048'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.138'MB, throughput = 4637.543'MB/sec zip my value time = 0.15's, unzip length = 0.013'GB zip my value throughput = 90.114'MB/sec zip pipeline throughput = 90.114'MB/sec entries = 112826 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.56 avg-zval = 41.40 usrkeys = 112826 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.56 avg-zval = 41.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 112826 bytes = 352592 } seq expand size = 789782 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0023 GB value = 0.0133 GB dict = 0.14 MB all = 0.0164 GB } __ZipSize{ index = 0.0002 GB value = 0.0047 GB dict = 0.14 MB all = 0.0054 GB } UnZip/Zip{ index = 10.3823 value = 2.8398 dict = 1.00 all = 3.0525 } Zip/UnZip{ index = 0.0963 value = 0.3521 dict = 1.00 all = 0.3276 } ---------------------------- total value len = 11.029968 GB avg = 0.157 KB (by entry num) total key len = 1.937719 GB avg = 0.028 KB total ukey len = 1.352805 GB avg = 0.016 KB total ukey num = 0.083296791 Billion total entry num = 0.070104520 Billion write speed all = 0.376436757 MB/sec (with seq num) write speed all = 0.360156340 MB/sec (without seq num) 2024/11/23-18:29:20.741240 139957592835648 [compaction_job.cc:1948] [default] [JOB 33103] Generated table #274866: 112826 keys, 5380131 bytes, temperature: kWarm 2024/11/23-18:29:20.741309 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357760741276, "cf_name": "default", "job": 33103, "event": "table_file_creation", "file_number": 274866, "file_size": 5380131, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4164904669, "largest_seqno": 4183069114, "table_properties": {"data_size": 4670592, "index_size": 217344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3159128, "raw_average_key_size": 28, "raw_value_size": 13263612, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 112826, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357760, "oldest_key_time": 0, "file_creation_time": 1732357760, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274866, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:29:20.751914 139957592835648 [compaction_job.cc:703] job-33103: subcompact[0], size: 0.732543 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33103/att-00 [274847,274848,274849,274850,274851,274852,274853,274854,274855,274856,274857,274858,274859,274860,274861,274862,274863,274864,274865,274866] 2024/11/23-18:29:20.868502 139957592835648 (Original Log Time 2024/11/23-18:28:14.019709) EVENT_LOG_v1 {"time_micros": 1732357694019644, "job": 33103, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [274816], "files_L3": [274766, 274767, 274768, 274769, 274770, 274771, 274772, 274773, 274774, 274775, 274776, 274778, 274780, 274782, 274783, 274785, 274786, 274787, 274839], "score": 0.989469, "input_data_size": 754600758, "oldest_snapshot_seqno": 4183315014} 2024/11/23-18:29:20.868506 139957592835648 (Original Log Time 2024/11/23-18:29:20.866921) [compaction_job.cc:2022] [default] [JOB 33103] Compacted 1@2 + 19@3 files to L3 => 732542702 bytes 2024/11/23-18:29:20.868507 139957592835648 (Original Log Time 2024/11/23-18:29:20.868432) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.3 rd, 11.0 wr, level 3, files in(1, 19) out(20 +0 blob) MB in(25.7, 693.9 +0.0 blob) out(698.6 +0.0 blob), read-write-amplify(55.1) write-amplify(27.1) OK, records in: 17854765, records dropped: 108409 output_compression: Snappy 2024/11/23-18:29:20.868510 139957592835648 (Original Log Time 2024/11/23-18:29:20.868477) EVENT_LOG_v1 {"time_micros": 1732357760868456, "cf": "default", "job": 33103, "event": "compaction_finished", "compaction_time_micros": 66732209, "compaction_time_cpu_micros": 27475718, "output_level": 3, "num_output_files": 20, "total_output_size": 732542702, "num_input_records": 17854765, "num_output_records": 17746356, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 503955329, "file_range_sync_nanos": 0, "file_fsync_nanos": 13925, "file_prepare_write_nanos": 51903, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-18:29:20.994915 139957592835648 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732357693,"executesMs":67010,"compactionJobId":33103,"attempt":0,"compactionInputRawBytes":2581955062,"compactionInputZipBytes":754600758,"compactionOutputRawBytes":0,"compactionOutputZipBytes":732542702,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b024f7fcfb4c2e5f7b5e3c11210c4701-49a8ffd518f3cac2-00"}}, response = 2024/11/23-18:29:20.995032 139957592835648 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33103/att-00: olev 3, work 67.011 s, result 7.811 ms, install 1.603 ms, input{raw 2.405 GiB zip 719.643 MiB}