2024/11/23-14:09:13.800652 139957416588864 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00: fork to child time = 0.008716 sec 2024/11/23-14:09:13.801465 139957416588864 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00 2024/11/23-14:09:13.808410 139957416588864 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00 2024/11/23-14:09:13.809304 139957416588864 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:09:13.809555 139957416588864 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:09:13.873250 139957416588864 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27341/att-00/MANIFEST-206347 2024/11/23-14:09:13.894820 139957416588864 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27341/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 262387, last_sequence is 4010930897, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 262239 2024/11/23-14:09:13.894828 139957416588864 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:09:13.894939 139957416588864 [version_set.cc:5721] Creating manifest 262387 2024/11/23-14:09:13.932310 139957416588864 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:09:13.951383 139957416588864 [compaction_job.cc:2437] [default] [JOB 27341] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-14:09:13.951400 139957416588864 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20478 Base level 2, inputs: [262383(26MB)], [262340(34MB) 262341(36MB) 262342(35MB) 262343(35MB) 262344(34MB) 262345(36MB) 262346(34MB) 262347(33MB) 262348(34MB) 262349(36MB) 262350(35MB) 262351(35MB) 262352(35MB) 262353(35MB) 262354(25MB)] 2024/11/23-14:09:13.959468 139957416588864 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:09:13.959498 139957416588864 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:09:13.959517 139957416588864 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:09:13.959523 139957416588864 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:09:13.959534 139957416588864 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:09:14.501650 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262388.sst) /dev/shm/ToplingTemp/Topling-67916-my26UG 2024/11/23-14:09:15.361807 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262388.sst): first pass time = 0.86's, 156.046'MB/sec 2024/11/23-14:09:17.426668 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262388.sst): old prealloc_size = 147639500, real_size = 31636608 2024/11/23-14:09:17.427458 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262388.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932008 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 116.01 avg-zval = 31.78 2024/11/23-14:09:17.428349 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262388.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:18.315571 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262388.sst): second pass time = 2.06's, 52.559'MB/sec, value only(80.6% of KV) wait indexing time = 0.01's, remap KeyValue time = 0.89's, 40.349'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2332.722'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.17's, 213.964'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29697.523'MB/sec zip my value time = 2.06's, unzip length = 0.108'GB zip my value throughput = 52.559'MB/sec zip pipeline throughput = 57.028'MB/sec entries = 932008 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.01 avg-zval = 31.78 usrkeys = 932008 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.01 avg-zval = 31.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932008 bytes = 3378544 } seq expand size = 6524056 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.4466 value = 3.6504 dict = 1.00 all = 3.7422 } Zip/UnZip{ index = 0.0957 value = 0.2739 dict = 1.00 all = 0.2672 } ---------------------------- total value len = 8.963265 GB avg = 0.168 KB (by entry num) total key len = 1.466917 GB avg = 0.028 KB total ukey len = 1.016518 GB avg = 0.015 KB total ukey num = 0.066482443 Billion total entry num = 0.053290172 Billion write speed all = 0.553439820 MB/sec (with seq num) write speed all = 0.530818623 MB/sec (without seq num) 2024/11/23-14:09:18.364767 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262388: 932008 keys, 35865618 bytes, temperature: kWarm 2024/11/23-14:09:18.364881 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342158364810, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262388, "file_size": 35865618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 4010914730, "table_properties": {"data_size": 29619120, "index_size": 1784320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26096224, "raw_average_key_size": 28, "raw_value_size": 108121548, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 932008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342154, "oldest_key_time": 0, "file_creation_time": 1732342154, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262388, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:18.434475 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262389.sst) /dev/shm/ToplingTemp/Topling-67916-DGiVnI 2024/11/23-14:09:19.732992 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262389.sst): first pass time = 1.30's, 103.367'MB/sec 2024/11/23-14:09:21.790498 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262389.sst): old prealloc_size = 147639500, real_size = 33371376 2024/11/23-14:09:21.791089 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262389.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931128 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0314 GB avg-val = 116.15 avg-zval = 33.67 2024/11/23-14:09:21.791934 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262389.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:22.997051 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262389.sst): second pass time = 2.06's, 52.609'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.21's, 31.259'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3122.214'MB/sec (index lex order gen) rebuild zvType time = 1.07's, 0.218'MB/sec write SST data time = 0.14's, 270.194'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28753.493'MB/sec zip my value time = 2.06's, unzip length = 0.108'GB zip my value throughput = 52.609'MB/sec zip pipeline throughput = 65.143'MB/sec entries = 931128 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 33.67 usrkeys = 931128 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 33.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931128 bytes = 3491744 } seq expand size = 6517896 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4450 value = 3.4490 dict = 1.00 all = 3.5588 } Zip/UnZip{ index = 0.0957 value = 0.2899 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 9.077930 GB avg = 0.167 KB (by entry num) total key len = 1.492989 GB avg = 0.028 KB total ukey len = 1.035141 GB avg = 0.015 KB total ukey num = 0.067413571 Billion total entry num = 0.054221300 Billion write speed all = 0.560768154 MB/sec (with seq num) write speed all = 0.537757417 MB/sec (without seq num) 2024/11/23-14:09:23.060673 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262389: 931128 keys, 37714368 bytes, temperature: kWarm 2024/11/23-14:09:23.060758 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342163060711, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262389, "file_size": 37714368, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4010913920, "table_properties": {"data_size": 31355488, "index_size": 1782912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26071584, "raw_average_key_size": 28, "raw_value_size": 108146248, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931128, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342158, "oldest_key_time": 0, "file_creation_time": 1732342158, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262389, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:23.135228 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262390.sst) /dev/shm/ToplingTemp/Topling-67916-e2U4NI 2024/11/23-14:09:24.103822 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262390.sst): first pass time = 0.97's, 138.575'MB/sec 2024/11/23-14:09:25.640411 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262390.sst): old prealloc_size = 147639500, real_size = 32493056 2024/11/23-14:09:25.641023 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262390.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931347 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 116.11 avg-zval = 32.74 2024/11/23-14:09:25.641837 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262390.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:26.511065 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262390.sst): second pass time = 1.54's, 70.413'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.043'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3004.394'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 277.292'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28390.912'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.413'MB/sec zip pipeline throughput = 70.413'MB/sec entries = 931347 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.11 avg-zval = 32.74 usrkeys = 931347 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.11 avg-zval = 32.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931347 bytes = 3259728 } seq expand size = 6519429 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5135 value = 3.5469 dict = 1.00 all = 3.6667 } Zip/UnZip{ index = 0.0951 value = 0.2819 dict = 1.00 all = 0.2727 } ---------------------------- total value len = 9.192589 GB avg = 0.167 KB (by entry num) total key len = 1.519067 GB avg = 0.028 KB total ukey len = 1.053768 GB avg = 0.015 KB total ukey num = 0.068344918 Billion total entry num = 0.055152647 Billion write speed all = 0.568128103 MB/sec (with seq num) write speed all = 0.544726478 MB/sec (without seq num) 2024/11/23-14:09:26.556969 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262390: 931347 keys, 36604148 bytes, temperature: kWarm 2024/11/23-14:09:26.557053 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342166557019, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262390, "file_size": 36604148, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4010914820, "table_properties": {"data_size": 30488304, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26077716, "raw_average_key_size": 28, "raw_value_size": 108140086, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931347, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342163, "oldest_key_time": 0, "file_creation_time": 1732342163, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262390, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:26.596670 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262391.sst) /dev/shm/ToplingTemp/Topling-67916-YDIHJE 2024/11/23-14:09:27.748044 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262391.sst): first pass time = 1.15's, 116.576'MB/sec 2024/11/23-14:09:29.351538 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262391.sst): old prealloc_size = 147639500, real_size = 33089968 2024/11/23-14:09:29.352116 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262391.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930371 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.26 avg-zval = 33.41 2024/11/23-14:09:29.352897 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262391.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:30.235248 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262391.sst): second pass time = 1.60's, 67.490'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.094'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3182.161'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 285.952'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 40749.509'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.490'MB/sec zip pipeline throughput = 69.936'MB/sec entries = 930371 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.26 avg-zval = 33.41 usrkeys = 930371 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.26 avg-zval = 33.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930371 bytes = 3256320 } seq expand size = 6512597 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5150 value = 3.4794 dict = 1.00 all = 3.6082 } Zip/UnZip{ index = 0.0951 value = 0.2874 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 9.307269 GB avg = 0.166 KB (by entry num) total key len = 1.545117 GB avg = 0.028 KB total ukey len = 1.072375 GB avg = 0.015 KB total ukey num = 0.069275289 Billion total entry num = 0.056083018 Billion write speed all = 0.575478531 MB/sec (with seq num) write speed all = 0.551686843 MB/sec (without seq num) 2024/11/23-14:09:30.281214 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262391: 930371 keys, 37198126 bytes, temperature: kWarm 2024/11/23-14:09:30.281493 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342170281460, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262391, "file_size": 37198126, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 4010914947, "table_properties": {"data_size": 31087584, "index_size": 1769600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050388, "raw_average_key_size": 28, "raw_value_size": 108167354, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930371, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342166, "oldest_key_time": 0, "file_creation_time": 1732342166, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262391, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:30.318167 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262392.sst) /dev/shm/ToplingTemp/Topling-67916-Ajo6tG 2024/11/23-14:09:31.355158 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262392.sst): first pass time = 1.04's, 129.435'MB/sec 2024/11/23-14:09:33.013925 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262392.sst): old prealloc_size = 147639500, real_size = 34149456 2024/11/23-14:09:33.014520 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262392.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935290 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0321 GB avg-val = 115.50 avg-zval = 34.35 2024/11/23-14:09:33.014945 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262392.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:33.795755 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262392.sst): second pass time = 1.66's, 65.156'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 46.871'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3124.669'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.358'MB/sec write SST data time = 0.13's, 285.729'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35856.558'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.156'MB/sec zip pipeline throughput = 67.363'MB/sec entries = 935290 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.50 avg-zval = 34.35 usrkeys = 935290 avg-key = 20.00 avg-zkey = 1.92 avg-val = 118.82 avg-zval = 34.35 TagRS{ kinds = RS_Key0_TagN bytes = 146240 } TagArray{ size = 442803 bytes = 1494480 } seq expand size = 3099621 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4437 value = 3.3629 dict = 1.00 all = 3.6628 } Zip/UnZip{ index = 0.0958 value = 0.2974 dict = 1.00 all = 0.2730 } ---------------------------- total value len = 9.418398 GB avg = 0.165 KB (by entry num) total key len = 1.571305 GB avg = 0.028 KB total ukey len = 1.091081 GB avg = 0.016 KB total ukey num = 0.070210579 Billion total entry num = 0.057018308 Billion write speed all = 0.582650171 MB/sec (with seq num) write speed all = 0.558466277 MB/sec (without seq num) 2024/11/23-14:09:33.842874 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262392: 935290 keys, 36643915 bytes, temperature: kWarm 2024/11/23-14:09:33.842944 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342173842908, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262392, "file_size": 36643915, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4010914636, "table_properties": {"data_size": 32124352, "index_size": 1791104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26188120, "raw_average_key_size": 28, "raw_value_size": 108029708, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935290, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342170, "oldest_key_time": 0, "file_creation_time": 1732342170, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262392, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:33.879566 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262393.sst) /dev/shm/ToplingTemp/Topling-67916-tHy9rG 2024/11/23-14:09:34.819967 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262393.sst): first pass time = 0.94's, 142.729'MB/sec 2024/11/23-14:09:36.220408 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262393.sst): old prealloc_size = 147639500, real_size = 35453088 2024/11/23-14:09:36.221192 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262393.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080053 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1040 GB zip-val = 0.0331 GB avg-val = 96.27 avg-zval = 30.65 2024/11/23-14:09:36.221512 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262393.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:37.459092 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262393.sst): second pass time = 1.40's, 74.292'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.24's, 29.898'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2734.273'MB/sec (index lex order gen) rebuild zvType time = 1.07's, 0.252'MB/sec write SST data time = 0.17's, 224.284'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.026'MB, throughput = 32346.366'MB/sec zip my value time = 1.40's, unzip length = 0.104'GB zip my value throughput = 74.292'MB/sec zip pipeline throughput = 74.292'MB/sec entries = 1080053 avg-key = 28.00 avg-zkey = 1.93 avg-val = 96.27 avg-zval = 30.65 usrkeys = 1080053 avg-key = 20.00 avg-zkey = 1.93 avg-val = 97.46 avg-zval = 30.65 TagRS{ kinds = RS_Key0_TagN bytes = 168840 } TagArray{ size = 184137 bytes = 644496 } seq expand size = 1288959 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0331 GB dict = 1.03 MB all = 0.0370 GB } UnZip/Zip{ index = 10.3797 value = 3.1411 dict = 1.00 all = 3.6243 } Zip/UnZip{ index = 0.0963 value = 0.3184 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 9.523664 GB avg = 0.164 KB (by entry num) total key len = 1.601547 GB avg = 0.028 KB total ukey len = 1.112682 GB avg = 0.016 KB total ukey num = 0.071290632 Billion total entry num = 0.058098361 Billion write speed all = 0.589719908 MB/sec (with seq num) write speed all = 0.565082704 MB/sec (without seq num) 2024/11/23-14:09:37.523243 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262393: 1080053 keys, 37033112 bytes, temperature: kWarm 2024/11/23-14:09:37.523300 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342177523268, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262393, "file_size": 37033112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3416825712, "table_properties": {"data_size": 33101776, "index_size": 2081088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30241484, "raw_average_key_size": 28, "raw_value_size": 103976327, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080053, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342173, "oldest_key_time": 0, "file_creation_time": 1732342173, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262393, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:37.563729 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262394.sst) /dev/shm/ToplingTemp/Topling-67916-zgFieI 2024/11/23-14:09:38.921072 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262394.sst): first pass time = 1.36's, 98.885'MB/sec 2024/11/23-14:09:40.473120 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262394.sst): old prealloc_size = 147639500, real_size = 30703664 2024/11/23-14:09:40.473813 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262394.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931394 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0287 GB avg-val = 116.10 avg-zval = 30.80 2024/11/23-14:09:40.474725 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262394.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:41.710221 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262394.sst): second pass time = 1.55's, 69.711'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.24's, 28.331'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2658.616'MB/sec (index lex order gen) rebuild zvType time = 1.08's, 0.215'MB/sec write SST data time = 0.15's, 229.266'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 23504.524'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.711'MB/sec zip pipeline throughput = 69.711'MB/sec entries = 931394 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.10 avg-zval = 30.80 usrkeys = 931394 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.10 avg-zval = 30.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931394 bytes = 3492736 } seq expand size = 6519758 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4458 value = 3.7696 dict = 1.00 all = 3.8296 } Zip/UnZip{ index = 0.0957 value = 0.2653 dict = 1.00 all = 0.2611 } ---------------------------- total value len = 9.638322 GB avg = 0.163 KB (by entry num) total key len = 1.627626 GB avg = 0.028 KB total ukey len = 1.131310 GB avg = 0.016 KB total ukey num = 0.072222026 Billion total entry num = 0.059029755 Billion write speed all = 0.597045521 MB/sec (with seq num) write speed all = 0.572018989 MB/sec (without seq num) 2024/11/23-14:09:41.755405 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262394: 931394 keys, 35047600 bytes, temperature: kWarm 2024/11/23-14:09:41.755474 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342181755439, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262394, "file_size": 35047600, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3416825713, "largest_seqno": 4010912532, "table_properties": {"data_size": 28687328, "index_size": 1783296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26079032, "raw_average_key_size": 28, "raw_value_size": 108138812, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931394, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342177, "oldest_key_time": 0, "file_creation_time": 1732342177, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262394, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:41.791510 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262395.sst) /dev/shm/ToplingTemp/Topling-67916-OLVQ7G 2024/11/23-14:09:42.988854 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262395.sst): first pass time = 1.20's, 112.099'MB/sec 2024/11/23-14:09:44.555507 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262395.sst): old prealloc_size = 147639500, real_size = 30534208 2024/11/23-14:09:44.556222 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262395.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931277 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0285 GB avg-val = 116.12 avg-zval = 30.63 2024/11/23-14:09:44.557099 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262395.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:45.538392 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262395.sst): second pass time = 1.57's, 69.098'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 35.249'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2558.396'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.15's, 231.637'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33632.190'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 69.098'MB/sec zip pipeline throughput = 69.098'MB/sec entries = 931277 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 30.63 usrkeys = 931277 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 30.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931277 bytes = 3259488 } seq expand size = 6518939 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.5124 value = 3.7905 dict = 1.00 all = 3.8741 } Zip/UnZip{ index = 0.0951 value = 0.2638 dict = 1.00 all = 0.2581 } ---------------------------- total value len = 9.752983 GB avg = 0.163 KB (by entry num) total key len = 1.653701 GB avg = 0.028 KB total ukey len = 1.149935 GB avg = 0.016 KB total ukey num = 0.073153303 Billion total entry num = 0.059961032 Billion write speed all = 0.604381329 MB/sec (with seq num) write speed all = 0.578965125 MB/sec (without seq num) 2024/11/23-14:09:45.582638 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262395: 931277 keys, 34644984 bytes, temperature: kWarm 2024/11/23-14:09:45.582722 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342185582675, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262395, "file_size": 34644984, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3798494465, "largest_seqno": 4010914866, "table_properties": {"data_size": 28529456, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26075756, "raw_average_key_size": 28, "raw_value_size": 108141974, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931277, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342181, "oldest_key_time": 0, "file_creation_time": 1732342181, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262395, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:45.626891 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262396.sst) /dev/shm/ToplingTemp/Topling-67916-3fLvEG 2024/11/23-14:09:46.588854 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262396.sst): first pass time = 0.96's, 139.534'MB/sec 2024/11/23-14:09:47.897047 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262396.sst): old prealloc_size = 147639500, real_size = 31101280 2024/11/23-14:09:47.897594 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262396.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931291 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0291 GB avg-val = 116.12 avg-zval = 31.24 2024/11/23-14:09:47.898419 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262396.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:48.722076 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262396.sst): second pass time = 1.31's, 82.735'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.963'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3370.286'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.12's, 283.721'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34445.765'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.735'MB/sec zip pipeline throughput = 82.735'MB/sec entries = 931291 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 31.24 usrkeys = 931291 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 31.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931291 bytes = 3492352 } seq expand size = 6519037 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5121 value = 3.7167 dict = 1.00 all = 3.7866 } Zip/UnZip{ index = 0.0951 value = 0.2691 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 9.867644 GB avg = 0.162 KB (by entry num) total key len = 1.679778 GB avg = 0.028 KB total ukey len = 1.168561 GB avg = 0.016 KB total ukey num = 0.074084594 Billion total entry num = 0.060892323 Billion write speed all = 0.611735056 MB/sec (with seq num) write speed all = 0.585928450 MB/sec (without seq num) 2024/11/23-14:09:48.766461 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262396: 931291 keys, 35445100 bytes, temperature: kWarm 2024/11/23-14:09:48.766531 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342188766494, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262396, "file_size": 35445100, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4010914549, "table_properties": {"data_size": 29096464, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076148, "raw_average_key_size": 28, "raw_value_size": 108141650, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931291, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342185, "oldest_key_time": 0, "file_creation_time": 1732342185, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262396, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:48.803366 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262397.sst) /dev/shm/ToplingTemp/Topling-67916-Av9o1F 2024/11/23-14:09:49.643921 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262397.sst): first pass time = 0.84's, 159.684'MB/sec 2024/11/23-14:09:50.957085 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262397.sst): old prealloc_size = 147639500, real_size = 32295216 2024/11/23-14:09:50.957618 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262397.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924822 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 117.13 avg-zval = 32.77 2024/11/23-14:09:50.958405 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262397.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:51.780499 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262397.sst): second pass time = 1.31's, 82.541'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.334'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3408.308'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.12's, 294.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33861.527'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.541'MB/sec zip pipeline throughput = 82.541'MB/sec entries = 924822 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.13 avg-zval = 32.77 usrkeys = 924822 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.13 avg-zval = 32.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924822 bytes = 3352496 } seq expand size = 6473754 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5139 value = 3.5745 dict = 1.00 all = 3.6768 } Zip/UnZip{ index = 0.0951 value = 0.2798 dict = 1.00 all = 0.2720 } ---------------------------- total value len = 9.982440 GB avg = 0.161 KB (by entry num) total key len = 1.705673 GB avg = 0.028 KB total ukey len = 1.187058 GB avg = 0.016 KB total ukey num = 0.075009416 Billion total entry num = 0.061817145 Billion write speed all = 0.619088015 MB/sec (with seq num) write speed all = 0.592893707 MB/sec (without seq num) 2024/11/23-14:09:51.825078 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262397: 924822 keys, 36503686 bytes, temperature: kWarm 2024/11/23-14:09:51.825146 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342191825112, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262397, "file_size": 36503686, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3607727220, "largest_seqno": 4010913324, "table_properties": {"data_size": 30304608, "index_size": 1759232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25895016, "raw_average_key_size": 28, "raw_value_size": 108322848, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924822, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342188, "oldest_key_time": 0, "file_creation_time": 1732342188, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262397, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:51.861537 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262398.sst) /dev/shm/ToplingTemp/Topling-67916-lc1u6F 2024/11/23-14:09:52.865621 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262398.sst): first pass time = 1.00's, 133.676'MB/sec 2024/11/23-14:09:54.186813 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262398.sst): old prealloc_size = 147639500, real_size = 33855312 2024/11/23-14:09:54.187354 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262398.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922411 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.51 avg-zval = 34.54 2024/11/23-14:09:54.188142 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262398.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:55.041249 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262398.sst): second pass time = 1.32's, 82.086'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.542'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3408.133'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 303.382'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33116.559'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.086'MB/sec zip pipeline throughput = 82.086'MB/sec entries = 922411 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.51 avg-zval = 34.54 usrkeys = 922411 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.51 avg-zval = 34.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922411 bytes = 3343760 } seq expand size = 6456877 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4447 value = 3.4023 dict = 1.00 all = 3.5267 } Zip/UnZip{ index = 0.0957 value = 0.2939 dict = 1.00 all = 0.2836 } ---------------------------- total value len = 10.097288 GB avg = 0.161 KB (by entry num) total key len = 1.731500 GB avg = 0.028 KB total ukey len = 1.205506 GB avg = 0.016 KB total ukey num = 0.075931827 Billion total entry num = 0.062739556 Billion write speed all = 0.626430979 MB/sec (with seq num) write speed all = 0.599850401 MB/sec (without seq num) 2024/11/23-14:09:55.087903 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262398: 922411 keys, 38057630 bytes, temperature: kWarm 2024/11/23-14:09:55.087973 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342195087936, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262398, "file_size": 38057630, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4010914702, "table_properties": {"data_size": 31858240, "index_size": 1766272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25827508, "raw_average_key_size": 28, "raw_value_size": 108390234, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922411, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342191, "oldest_key_time": 0, "file_creation_time": 1732342191, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262398, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:55.123700 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262399.sst) /dev/shm/ToplingTemp/Topling-67916-WlcD8H 2024/11/23-14:09:56.152073 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262399.sst): first pass time = 1.03's, 130.519'MB/sec 2024/11/23-14:09:57.457531 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262399.sst): old prealloc_size = 147639500, real_size = 32847952 2024/11/23-14:09:57.458091 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262399.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931281 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.12 avg-zval = 33.11 2024/11/23-14:09:57.458885 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262399.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:09:58.315693 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262399.sst): second pass time = 1.30's, 82.887'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.204'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3338.131'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 298.015'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33839.829'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.887'MB/sec zip pipeline throughput = 82.887'MB/sec entries = 931281 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.12 avg-zval = 33.11 usrkeys = 931281 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.12 avg-zval = 33.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931281 bytes = 3375904 } seq expand size = 6518967 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4441 value = 3.5075 dict = 1.00 all = 3.6201 } Zip/UnZip{ index = 0.0957 value = 0.2851 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 10.211948 GB avg = 0.160 KB (by entry num) total key len = 1.757576 GB avg = 0.028 KB total ukey len = 1.224131 GB avg = 0.016 KB total ukey num = 0.076863108 Billion total entry num = 0.063670837 Billion write speed all = 0.633774237 MB/sec (with seq num) write speed all = 0.606803785 MB/sec (without seq num) 2024/11/23-14:09:58.361276 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262399: 931281 keys, 37075220 bytes, temperature: kWarm 2024/11/23-14:09:58.361354 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342198361308, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262399, "file_size": 37075220, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 4010912223, "table_properties": {"data_size": 30831616, "index_size": 1783360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26075868, "raw_average_key_size": 28, "raw_value_size": 108141890, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931281, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342195, "oldest_key_time": 0, "file_creation_time": 1732342195, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262399, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:09:58.397457 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262400.sst) /dev/shm/ToplingTemp/Topling-67916-nWZNNI 2024/11/23-14:09:59.255231 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262400.sst): first pass time = 0.86's, 156.480'MB/sec 2024/11/23-14:10:00.556503 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262400.sst): old prealloc_size = 147639500, real_size = 32312032 2024/11/23-14:10:00.557049 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262400.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931497 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 116.09 avg-zval = 32.54 2024/11/23-14:10:00.557862 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262400.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:10:01.405519 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262400.sst): second pass time = 1.30's, 83.149'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.039'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3374.810'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 291.468'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35301.247'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.149'MB/sec zip pipeline throughput = 83.149'MB/sec entries = 931497 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 32.54 usrkeys = 931497 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 32.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931497 bytes = 3376688 } seq expand size = 6520479 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5118 value = 3.5681 dict = 1.00 all = 3.6732 } Zip/UnZip{ index = 0.0951 value = 0.2803 dict = 1.00 all = 0.2722 } ---------------------------- total value len = 10.326605 GB avg = 0.160 KB (by entry num) total key len = 1.783658 GB avg = 0.028 KB total ukey len = 1.242761 GB avg = 0.016 KB total ukey num = 0.077794605 Billion total entry num = 0.064602334 Billion write speed all = 0.641121299 MB/sec (with seq num) write speed all = 0.613760748 MB/sec (without seq num) 2024/11/23-14:10:01.450408 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262400: 931497 keys, 36539854 bytes, temperature: kWarm 2024/11/23-14:10:01.450479 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342201450443, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262400, "file_size": 36539854, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4010914464, "table_properties": {"data_size": 30306704, "index_size": 1772288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26081916, "raw_average_key_size": 28, "raw_value_size": 108135914, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931497, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342198, "oldest_key_time": 0, "file_creation_time": 1732342198, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262400, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:10:01.492270 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262401.sst) /dev/shm/ToplingTemp/Topling-67916-NP5FrI 2024/11/23-14:10:02.534504 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262401.sst): first pass time = 1.04's, 128.784'MB/sec 2024/11/23-14:10:03.837093 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262401.sst): old prealloc_size = 147639500, real_size = 33415008 2024/11/23-14:10:03.837647 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262401.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927392 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.73 avg-zval = 33.88 2024/11/23-14:10:03.838478 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262401.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:10:04.700429 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262401.sst): second pass time = 1.30's, 83.158'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.724'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.633'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.315'MB/sec write SST data time = 0.13's, 295.182'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33106.768'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.158'MB/sec zip pipeline throughput = 83.158'MB/sec entries = 927392 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 33.88 usrkeys = 927392 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 33.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927392 bytes = 3477728 } seq expand size = 6491744 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5156 value = 3.4454 dict = 1.00 all = 3.5557 } Zip/UnZip{ index = 0.0951 value = 0.2902 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 10.441347 GB avg = 0.159 KB (by entry num) total key len = 1.809625 GB avg = 0.028 KB total ukey len = 1.261309 GB avg = 0.016 KB total ukey num = 0.078721997 Billion total entry num = 0.065529726 Billion write speed all = 0.648457399 MB/sec (with seq num) write speed all = 0.620708917 MB/sec (without seq num) 2024/11/23-14:10:04.745788 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262401: 927392 keys, 37747342 bytes, temperature: kWarm 2024/11/23-14:10:04.745856 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342204745822, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262401, "file_size": 37747342, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4010913025, "table_properties": {"data_size": 31419152, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966976, "raw_average_key_size": 28, "raw_value_size": 108250880, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927392, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342201, "oldest_key_time": 0, "file_creation_time": 1732342201, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262401, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:10:04.782347 139957416588864 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262402.sst) /dev/shm/ToplingTemp/Topling-67916-dKQ0NG 2024/11/23-14:10:05.601711 139957416588864 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262402.sst): first pass time = 0.82's, 156.676'MB/sec 2024/11/23-14:10:06.841690 139957416588864 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262402.sst): old prealloc_size = 147639500, real_size = 30278208 2024/11/23-14:10:06.842218 139957416588864 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262402.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 892654 prefix = 4 raw-key = 0.0143 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1034 GB zip-val = 0.0284 GB avg-val = 115.80 avg-zval = 31.77 2024/11/23-14:10:06.842863 139957416588864 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262402.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0022 GB, waited 0.000 sec, Key+Value = 0.128 GB 2024/11/23-14:10:07.645147 139957416588864 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00/262402.sst): second pass time = 1.24's, 83.444'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 42.021'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3374.357'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.329'MB/sec write SST data time = 0.12's, 270.174'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.035'MB, throughput = 32191.991'MB/sec zip my value time = 1.24's, unzip length = 0.103'GB zip my value throughput = 83.444'MB/sec zip pipeline throughput = 83.444'MB/sec entries = 892654 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.80 avg-zval = 31.77 usrkeys = 892654 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.66 avg-zval = 31.77 TagRS{ kinds = RS_Key0_TagN bytes = 139560 } TagArray{ size = 747465 bytes = 2522704 } seq expand size = 5232255 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0179 GB value = 0.1034 GB dict = 1.03 MB all = 0.1284 GB } __ZipSize{ index = 0.0017 GB value = 0.0284 GB dict = 1.03 MB all = 0.0338 GB } UnZip/Zip{ index = 10.5119 value = 3.6455 dict = 1.00 all = 3.8022 } Zip/UnZip{ index = 0.0951 value = 0.2743 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 10.549952 GB avg = 0.159 KB (by entry num) total key len = 1.834619 GB avg = 0.028 KB total ukey len = 1.279162 GB avg = 0.016 KB total ukey num = 0.079614651 Billion total entry num = 0.066422380 Billion write speed all = 0.655426785 MB/sec (with seq num) write speed all = 0.627304692 MB/sec (without seq num) 2024/11/23-14:10:07.687204 139957416588864 [compaction_job.cc:1948] [default] [JOB 27341] Generated table #262402: 892654 keys, 33761226 bytes, temperature: kWarm 2024/11/23-14:10:07.687270 139957416588864 EVENT_LOG_v1 {"time_micros": 1732342207687236, "cf_name": "default", "job": 27341, "event": "table_file_creation", "file_number": 262402, "file_size": 33761226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4010299508, "table_properties": {"data_size": 28356464, "index_size": 1698368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 24994312, "raw_average_key_size": 28, "raw_value_size": 103372492, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 892654, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342204, "oldest_key_time": 0, "file_creation_time": 1732342204, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262402, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:10:07.711916 139957416588864 [compaction_job.cc:703] job-27341: subcompact[0], size: 0.545882 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27341/att-00 [262388,262389,262390,262391,262392,262393,262394,262395,262396,262397,262398,262399,262400,262401,262402] 2024/11/23-14:10:07.805815 139957416588864 (Original Log Time 2024/11/23-14:09:13.951470) EVENT_LOG_v1 {"time_micros": 1732342153951427, "job": 27341, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [262383], "files_L3": [262340, 262341, 262342, 262343, 262344, 262345, 262346, 262347, 262348, 262349, 262350, 262351, 262352, 262353, 262354], "score": 0.999864, "input_data_size": 571623980, "oldest_snapshot_seqno": 4010930832} 2024/11/23-14:10:07.805819 139957416588864 (Original Log Time 2024/11/23-14:10:07.803566) [compaction_job.cc:2022] [default] [JOB 27341] Compacted 1@2 + 15@3 files to L3 => 545881929 bytes 2024/11/23-14:10:07.805821 139957416588864 (Original Log Time 2024/11/23-14:10:07.804720) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.6 rd, 10.2 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(26.4, 518.7 +0.0 blob) out(520.6 +0.0 blob), read-write-amplify(40.3) write-amplify(19.7) OK, records in: 14105803, records dropped: 41587 output_compression: Snappy 2024/11/23-14:10:07.805828 139957416588864 (Original Log Time 2024/11/23-14:10:07.805791) EVENT_LOG_v1 {"time_micros": 1732342207804744, "cf": "default", "job": 27341, "event": "compaction_finished", "compaction_time_micros": 53760457, "compaction_time_cpu_micros": 22831163, "output_level": 3, "num_output_files": 15, "total_output_size": 545881929, "num_input_records": 14105803, "num_output_records": 14064216, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 402406455, "file_range_sync_nanos": 0, "file_fsync_nanos": 11338, "file_prepare_write_nanos": 39302, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-14:10:07.930479 139957416588864 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732342153,"executesMs":53994,"compactionJobId":27341,"attempt":0,"compactionInputRawBytes":2013321367,"compactionInputZipBytes":571623980,"compactionOutputRawBytes":0,"compactionOutputZipBytes":545881929,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b5d7d3b3b14e66e1949245a6bf24b454-159e512c9c8a8037-00"}}, response = 2024/11/23-14:10:07.930573 139957416588864 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27341/att-00: olev 3, work 53.995 s, result 8.166 ms, install 2.285 ms, input{raw 1.875 GiB zip 545.143 MiB}