2024/11/22-23:51:10.632282 140025205032512 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00: fork to child time = 0.009845 sec 2024/11/22-23:51:10.633110 140025205032512 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00 2024/11/22-23:51:10.640718 140025205032512 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00 2024/11/22-23:51:10.646084 140025205032512 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-23:51:10.646359 140025205032512 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-23:51:10.698887 140025205032512 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-14454/att-00/MANIFEST-206347 2024/11/22-23:51:10.727630 140025205032512 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-14454/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 234795, last_sequence is 3655408668, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 234656 2024/11/22-23:51:10.727638 140025205032512 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-23:51:10.727733 140025205032512 [version_set.cc:5721] Creating manifest 234795 2024/11/22-23:51:10.767621 140025205032512 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-23:51:10.792167 140025205032512 [compaction_job.cc:2437] [default] [JOB 14454] Compacting 1@2 + 14@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-23:51:10.792185 140025205032512 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 10832 Base level 2, inputs: [234789(20MB)], [234726(33MB) 234727(33MB) 234728(34MB) 234729(33MB) 234730(34MB) 234731(33MB) 234732(33MB) 234733(33MB) 234734(35MB) 234735(38MB) 234736(35MB) 234737(35MB) 234738(34MB) 234739(36MB)] 2024/11/22-23:51:10.797836 140025205032512 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:51:10.797885 140025205032512 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:51:10.797917 140025205032512 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:51:10.797926 140025205032512 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:51:10.797940 140025205032512 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:51:11.225187 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234796.sst) /dev/shm/ToplingTemp/Topling-339409-k3yLjG 2024/11/22-23:51:11.993820 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234796.sst): first pass time = 0.77's, 174.627'MB/sec 2024/11/22-23:51:13.682375 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234796.sst): old prealloc_size = 147639500, real_size = 31216416 2024/11/22-23:51:13.683088 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234796.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925419 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0292 GB avg-val = 117.03 avg-zval = 31.58 2024/11/22-23:51:13.683931 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234796.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:14.879806 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234796.sst): second pass time = 1.69's, 64.174'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.20's, 29.489'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2537.950'MB/sec (index lex order gen) rebuild zvType time = 1.06's, 0.217'MB/sec write SST data time = 0.13's, 267.140'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35229.535'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.174'MB/sec zip pipeline throughput = 64.174'MB/sec entries = 925419 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.03 avg-zval = 31.58 usrkeys = 925419 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.03 avg-zval = 31.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925419 bytes = 3238976 } seq expand size = 6477933 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.5146 value = 3.7060 dict = 1.00 all = 3.8011 } Zip/UnZip{ index = 0.0951 value = 0.2698 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 0.114784 GB avg = 0.124 KB (by entry num) total key len = 0.025912 GB avg = 0.028 KB total ukey len = 0.018508 GB avg = 0.020 KB total ukey num = 0.000925419 Billion total entry num = 0.000925419 Billion write speed all = 0.002608772 MB/sec (with seq num) write speed all = 0.002471500 MB/sec (without seq num) 2024/11/22-23:51:14.914914 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234796: 925419 keys, 35310506 bytes, temperature: kWarm 2024/11/22-23:51:14.915003 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290674914949, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234796, "file_size": 35310506, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3513012344, "largest_seqno": 3655386612, "table_properties": {"data_size": 29224592, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911732, "raw_average_key_size": 28, "raw_value_size": 108306078, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925419, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290671, "oldest_key_time": 0, "file_creation_time": 1732290671, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234796, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:14.953675 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234797.sst) /dev/shm/ToplingTemp/Topling-339409-ZjB8AI 2024/11/22-23:51:16.651510 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234797.sst): first pass time = 1.70's, 79.054'MB/sec 2024/11/22-23:51:18.552180 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234797.sst): old prealloc_size = 147639500, real_size = 31227408 2024/11/22-23:51:18.552702 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234797.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925299 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0292 GB avg-val = 117.05 avg-zval = 31.60 2024/11/22-23:51:18.553472 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234797.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:19.771131 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234797.sst): second pass time = 1.90's, 57.023'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.22's, 28.977'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3509.044'MB/sec (index lex order gen) rebuild zvType time = 1.01's, 0.230'MB/sec write SST data time = 0.21's, 165.726'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27708.945'MB/sec zip my value time = 1.90's, unzip length = 0.108'GB zip my value throughput = 57.023'MB/sec zip pipeline throughput = 57.023'MB/sec entries = 925299 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.05 avg-zval = 31.60 usrkeys = 925299 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.05 avg-zval = 31.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925299 bytes = 3238560 } seq expand size = 6477093 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.5132 value = 3.7047 dict = 1.00 all = 3.7999 } Zip/UnZip{ index = 0.0951 value = 0.2699 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 0.229571 GB avg = 0.124 KB (by entry num) total key len = 0.051820 GB avg = 0.028 KB total ukey len = 0.037014 GB avg = 0.020 KB total ukey num = 0.001850718 Billion total entry num = 0.001850718 Billion write speed all = 0.005217056 MB/sec (with seq num) write speed all = 0.004942553 MB/sec (without seq num) 2024/11/22-23:51:19.808827 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234797: 925299 keys, 35321196 bytes, temperature: kWarm 2024/11/22-23:51:19.808929 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290679808869, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234797, "file_size": 35321196, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3655386629, "table_properties": {"data_size": 29235648, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25908372, "raw_average_key_size": 28, "raw_value_size": 108309422, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925299, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290674, "oldest_key_time": 0, "file_creation_time": 1732290674, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234797, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:19.875579 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234798.sst) /dev/shm/ToplingTemp/Topling-339409-CpkKTG 2024/11/22-23:51:20.957582 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234798.sst): first pass time = 1.08's, 124.051'MB/sec 2024/11/22-23:51:22.804143 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234798.sst): old prealloc_size = 147639500, real_size = 32145856 2024/11/22-23:51:22.804665 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234798.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928371 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0301 GB avg-val = 116.57 avg-zval = 32.47 2024/11/22-23:51:22.805456 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234798.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:24.202699 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234798.sst): second pass time = 1.84's, 58.665'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.40's, 25.919'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3514.029'MB/sec (index lex order gen) rebuild zvType time = 1.10's, 0.212'MB/sec write SST data time = 0.30's, 120.183'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31486.801'MB/sec zip my value time = 1.84's, unzip length = 0.108'GB zip my value throughput = 58.665'MB/sec zip pipeline throughput = 80.567'MB/sec entries = 928371 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.57 avg-zval = 32.47 usrkeys = 928371 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.57 avg-zval = 32.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928371 bytes = 3249312 } seq expand size = 6498597 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5130 value = 3.5898 dict = 1.00 all = 3.7027 } Zip/UnZip{ index = 0.0951 value = 0.2786 dict = 1.00 all = 0.2701 } ---------------------------- total value len = 0.344293 GB avg = 0.124 KB (by entry num) total key len = 0.077814 GB avg = 0.028 KB total ukey len = 0.055582 GB avg = 0.020 KB total ukey num = 0.002779089 Billion total entry num = 0.002779089 Billion write speed all = 0.007825331 MB/sec (with seq num) write speed all = 0.007413165 MB/sec (without seq num) 2024/11/22-23:51:24.240877 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234798: 928371 keys, 36248374 bytes, temperature: kWarm 2024/11/22-23:51:24.240950 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290684240908, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234798, "file_size": 36248374, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3655386909, "table_properties": {"data_size": 30147440, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25994388, "raw_average_key_size": 28, "raw_value_size": 108223414, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928371, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290679, "oldest_key_time": 0, "file_creation_time": 1732290679, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234798, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:24.301812 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234799.sst) /dev/shm/ToplingTemp/Topling-339409-HFsmVG 2024/11/22-23:51:25.156013 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234799.sst): first pass time = 0.85's, 157.135'MB/sec 2024/11/22-23:51:27.065407 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234799.sst): old prealloc_size = 147639500, real_size = 30877872 2024/11/22-23:51:27.066001 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234799.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924378 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.20 avg-zval = 31.23 2024/11/22-23:51:27.066732 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234799.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:28.235394 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234799.sst): second pass time = 1.91's, 56.774'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.17's, 29.888'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3105.859'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.246'MB/sec write SST data time = 0.23's, 151.095'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31236.913'MB/sec zip my value time = 1.91's, unzip length = 0.108'GB zip my value throughput = 56.774'MB/sec zip pipeline throughput = 56.774'MB/sec entries = 924378 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.20 avg-zval = 31.23 usrkeys = 924378 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.20 avg-zval = 31.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924378 bytes = 3235344 } seq expand size = 6470646 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.3797 value = 3.7531 dict = 1.00 all = 3.8383 } Zip/UnZip{ index = 0.0963 value = 0.2664 dict = 1.00 all = 0.2605 } ---------------------------- total value len = 0.459098 GB avg = 0.124 KB (by entry num) total key len = 0.103697 GB avg = 0.028 KB total ukey len = 0.074069 GB avg = 0.020 KB total ukey num = 0.003703467 Billion total entry num = 0.003703467 Billion write speed all = 0.010432737 MB/sec (with seq num) write speed all = 0.009883517 MB/sec (without seq num) 2024/11/22-23:51:28.272307 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234799: 924378 keys, 34967988 bytes, temperature: kWarm 2024/11/22-23:51:28.272387 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290688272343, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234799, "file_size": 34967988, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434855072, "largest_seqno": 3655386678, "table_properties": {"data_size": 28865504, "index_size": 1781120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25882584, "raw_average_key_size": 28, "raw_value_size": 108335184, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924378, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290684, "oldest_key_time": 0, "file_creation_time": 1732290684, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234799, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:28.322414 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234800.sst) /dev/shm/ToplingTemp/Topling-339409-0ergmH 2024/11/22-23:51:29.400628 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234800.sst): first pass time = 1.08's, 124.487'MB/sec 2024/11/22-23:51:30.954419 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234800.sst): old prealloc_size = 147639500, real_size = 31801232 2024/11/22-23:51:30.954976 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234800.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926116 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0298 GB avg-val = 116.93 avg-zval = 32.19 2024/11/22-23:51:30.955723 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234800.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:31.963478 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234800.sst): second pass time = 1.55's, 69.748'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 35.576'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3257.136'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.23's, 157.486'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36859.472'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.748'MB/sec zip pipeline throughput = 71.432'MB/sec entries = 926116 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.93 avg-zval = 32.19 usrkeys = 926116 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.93 avg-zval = 32.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926116 bytes = 3241424 } seq expand size = 6482812 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5099 value = 3.6329 dict = 1.00 all = 3.7389 } Zip/UnZip{ index = 0.0951 value = 0.2753 dict = 1.00 all = 0.2675 } ---------------------------- total value len = 0.573868 GB avg = 0.124 KB (by entry num) total key len = 0.129628 GB avg = 0.028 KB total ukey len = 0.092592 GB avg = 0.020 KB total ukey num = 0.004629583 Billion total entry num = 0.004629583 Billion write speed all = 0.013040054 MB/sec (with seq num) write speed all = 0.012353539 MB/sec (without seq num) 2024/11/22-23:51:31.995383 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234800: 926116 keys, 35897622 bytes, temperature: kWarm 2024/11/22-23:51:31.995445 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290691995414, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234800, "file_size": 35897622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3655386693, "table_properties": {"data_size": 29807168, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931248, "raw_average_key_size": 28, "raw_value_size": 108286620, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926116, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290688, "oldest_key_time": 0, "file_creation_time": 1732290688, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234800, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:32.031122 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234801.sst) /dev/shm/ToplingTemp/Topling-339409-I3cekJ 2024/11/22-23:51:32.800267 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234801.sst): first pass time = 0.77's, 174.511'MB/sec 2024/11/22-23:51:34.152005 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234801.sst): old prealloc_size = 147639500, real_size = 31066416 2024/11/22-23:51:34.152531 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234801.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922410 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.51 avg-zval = 31.51 2024/11/22-23:51:34.153306 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234801.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:35.165772 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234801.sst): second pass time = 1.35's, 80.273'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 34.675'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3497.210'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.261'MB/sec write SST data time = 0.13's, 270.222'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33982.559'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.273'MB/sec zip pipeline throughput = 80.273'MB/sec entries = 922410 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.51 avg-zval = 31.51 usrkeys = 922410 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.51 avg-zval = 31.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922410 bytes = 3228448 } seq expand size = 6456870 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4443 value = 3.7287 dict = 1.00 all = 3.8183 } Zip/UnZip{ index = 0.0957 value = 0.2682 dict = 1.00 all = 0.2619 } ---------------------------- total value len = 0.688715 GB avg = 0.124 KB (by entry num) total key len = 0.155456 GB avg = 0.028 KB total ukey len = 0.111040 GB avg = 0.020 KB total ukey num = 0.005551993 Billion total entry num = 0.005551993 Billion write speed all = 0.015646682 MB/sec (with seq num) write speed all = 0.014823433 MB/sec (without seq num) 2024/11/22-23:51:35.201581 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234801: 922410 keys, 35151570 bytes, temperature: kWarm 2024/11/22-23:51:35.201673 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290695201619, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234801, "file_size": 35151570, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3441224403, "largest_seqno": 3655386709, "table_properties": {"data_size": 29069280, "index_size": 1766336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25827480, "raw_average_key_size": 28, "raw_value_size": 108390388, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922410, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290692, "oldest_key_time": 0, "file_creation_time": 1732290692, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234801, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:35.251303 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234802.sst) /dev/shm/ToplingTemp/Topling-339409-cQEjNH 2024/11/22-23:51:36.313721 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234802.sst): first pass time = 1.06's, 126.338'MB/sec 2024/11/22-23:51:37.827065 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234802.sst): old prealloc_size = 147639500, real_size = 31965056 2024/11/22-23:51:37.827753 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234802.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926950 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0300 GB avg-val = 116.80 avg-zval = 32.32 2024/11/22-23:51:37.828567 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234802.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:38.866415 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234802.sst): second pass time = 1.51's, 71.575'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 34.587'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2658.956'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.263'MB/sec write SST data time = 0.16's, 228.333'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34129.100'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.575'MB/sec zip pipeline throughput = 71.575'MB/sec entries = 926950 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.80 avg-zval = 32.32 usrkeys = 926950 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.80 avg-zval = 32.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926950 bytes = 3128464 } seq expand size = 6488650 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.4458 value = 3.6138 dict = 1.00 all = 3.7337 } Zip/UnZip{ index = 0.0957 value = 0.2767 dict = 1.00 all = 0.2678 } ---------------------------- total value len = 0.803467 GB avg = 0.124 KB (by entry num) total key len = 0.181410 GB avg = 0.028 KB total ukey len = 0.129579 GB avg = 0.020 KB total ukey num = 0.006478943 Billion total entry num = 0.006478943 Billion write speed all = 0.018253418 MB/sec (with seq num) write speed all = 0.017292788 MB/sec (without seq num) 2024/11/22-23:51:38.898948 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234802: 926950 keys, 35947850 bytes, temperature: kWarm 2024/11/22-23:51:38.899006 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290698898975, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234802, "file_size": 35947850, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3522904895, "largest_seqno": 3655386722, "table_properties": {"data_size": 29958320, "index_size": 1774784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954600, "raw_average_key_size": 28, "raw_value_size": 108263144, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926950, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290695, "oldest_key_time": 0, "file_creation_time": 1732290695, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234802, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:38.934039 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234803.sst) /dev/shm/ToplingTemp/Topling-339409-wIjkdI 2024/11/22-23:51:40.064473 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234803.sst): first pass time = 1.13's, 118.735'MB/sec 2024/11/22-23:51:41.409720 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234803.sst): old prealloc_size = 147639500, real_size = 31930320 2024/11/22-23:51:41.410286 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234803.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920750 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0299 GB avg-val = 117.77 avg-zval = 32.51 2024/11/22-23:51:41.411043 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234803.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:42.296425 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234803.sst): second pass time = 1.34's, 80.653'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.615'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3229.306'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.303'MB/sec write SST data time = 0.13's, 283.069'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 36839.988'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.653'MB/sec zip pipeline throughput = 80.653'MB/sec entries = 920750 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.77 avg-zval = 32.51 usrkeys = 920750 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.77 avg-zval = 32.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920750 bytes = 3222640 } seq expand size = 6445250 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4448 value = 3.6222 dict = 1.00 all = 3.7270 } Zip/UnZip{ index = 0.0957 value = 0.2761 dict = 1.00 all = 0.2683 } ---------------------------- total value len = 0.918349 GB avg = 0.124 KB (by entry num) total key len = 0.207191 GB avg = 0.028 KB total ukey len = 0.147994 GB avg = 0.020 KB total ukey num = 0.007399693 Billion total entry num = 0.007399693 Billion write speed all = 0.020859099 MB/sec (with seq num) write speed all = 0.019762020 MB/sec (without seq num) 2024/11/22-23:51:42.328908 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234803: 920750 keys, 36012756 bytes, temperature: kWarm 2024/11/22-23:51:42.328970 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290702328939, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234803, "file_size": 36012756, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3655384391, "table_properties": {"data_size": 29936896, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25781000, "raw_average_key_size": 28, "raw_value_size": 108436820, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920750, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290698, "oldest_key_time": 0, "file_creation_time": 1732290698, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234803, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:42.366862 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234804.sst) /dev/shm/ToplingTemp/Topling-339409-Qa3JAH 2024/11/22-23:51:43.283804 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234804.sst): first pass time = 0.92's, 146.383'MB/sec 2024/11/22-23:51:44.638295 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234804.sst): old prealloc_size = 147639500, real_size = 32983312 2024/11/22-23:51:44.638877 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234804.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925342 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.05 avg-zval = 33.49 2024/11/22-23:51:44.640650 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234804.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:45.612020 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234804.sst): second pass time = 1.35's, 80.004'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.078'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3142.213'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.278'MB/sec write SST data time = 0.14's, 265.733'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33793.112'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.004'MB/sec zip pipeline throughput = 80.004'MB/sec entries = 925342 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.05 avg-zval = 33.49 usrkeys = 925342 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.05 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925342 bytes = 3238704 } seq expand size = 6477394 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5130 value = 3.4948 dict = 1.00 all = 3.6200 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 1.033135 GB avg = 0.124 KB (by entry num) total key len = 0.233101 GB avg = 0.028 KB total ukey len = 0.166501 GB avg = 0.020 KB total ukey num = 0.008325035 Billion total entry num = 0.008325035 Billion write speed all = 0.023465094 MB/sec (with seq num) write speed all = 0.022230899 MB/sec (without seq num) 2024/11/22-23:51:45.645445 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234804: 925342 keys, 37076828 bytes, temperature: kWarm 2024/11/22-23:51:45.645517 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290705645477, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234804, "file_size": 37076828, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3499102480, "largest_seqno": 3655386734, "table_properties": {"data_size": 30991424, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25909576, "raw_average_key_size": 28, "raw_value_size": 108308208, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925342, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290702, "oldest_key_time": 0, "file_creation_time": 1732290702, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234804, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:45.679845 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234805.sst) /dev/shm/ToplingTemp/Topling-339409-qpOGpG 2024/11/22-23:51:46.469450 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234805.sst): first pass time = 0.79's, 169.989'MB/sec 2024/11/22-23:51:48.022632 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234805.sst): old prealloc_size = 147639500, real_size = 36442896 2024/11/22-23:51:48.023179 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234805.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928896 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0344 GB avg-val = 116.49 avg-zval = 37.07 2024/11/22-23:51:48.023871 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234805.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:48.838736 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234805.sst): second pass time = 1.55's, 69.703'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 49.043'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3362.048'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 302.912'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34722.272'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.703'MB/sec zip pipeline throughput = 69.703'MB/sec entries = 928896 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.49 avg-zval = 37.07 usrkeys = 928896 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.85 avg-zval = 37.07 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 711361 bytes = 2578704 } seq expand size = 4979527 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4436 value = 3.1427 dict = 1.00 all = 3.3535 } Zip/UnZip{ index = 0.0958 value = 0.3182 dict = 1.00 all = 0.2982 } ---------------------------- total value len = 1.146323 GB avg = 0.124 KB (by entry num) total key len = 0.259110 GB avg = 0.028 KB total ukey len = 0.185079 GB avg = 0.020 KB total ukey num = 0.009253931 Billion total entry num = 0.009253931 Billion write speed all = 0.026043055 MB/sec (with seq num) write speed all = 0.024671231 MB/sec (without seq num) 2024/11/22-23:51:48.875151 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234805: 928896 keys, 40023590 bytes, temperature: kWarm 2024/11/22-23:51:48.875212 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290708875180, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234805, "file_size": 40023590, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3655386754, "table_properties": {"data_size": 34431616, "index_size": 1778880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009088, "raw_average_key_size": 28, "raw_value_size": 108208660, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928896, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290705, "oldest_key_time": 0, "file_creation_time": 1732290705, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234805, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:48.912720 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234806.sst) /dev/shm/ToplingTemp/Topling-339409-qZTIqI 2024/11/22-23:51:49.881547 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234806.sst): first pass time = 0.97's, 138.542'MB/sec 2024/11/22-23:51:51.497053 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234806.sst): old prealloc_size = 147639500, real_size = 36476048 2024/11/22-23:51:51.497613 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234806.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927006 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.79 avg-zval = 37.21 2024/11/22-23:51:51.497657 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234806.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:52.137477 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234806.sst): second pass time = 1.61's, 67.059'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.297'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3249.250'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.449'MB/sec write SST data time = 0.12's, 302.357'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35942.572'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.059'MB/sec zip pipeline throughput = 67.059'MB/sec entries = 927006 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.79 avg-zval = 37.21 usrkeys = 927006 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.79 avg-zval = 37.21 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5830 value = 3.1387 dict = 1.00 all = 3.5951 } Zip/UnZip{ index = 0.0945 value = 0.3186 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 1.254584 GB avg = 0.123 KB (by entry num) total key len = 0.285066 GB avg = 0.028 KB total ukey len = 0.203619 GB avg = 0.020 KB total ukey num = 0.010180937 Billion total entry num = 0.010180937 Billion write speed all = 0.028528402 MB/sec (with seq num) write speed all = 0.027019250 MB/sec (without seq num) 2024/11/22-23:51:52.169198 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234806: 927006 keys, 37334016 bytes, temperature: kWarm 2024/11/22-23:51:52.169261 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290712169229, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234806, "file_size": 37334016, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34492224, "index_size": 1751872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25956168, "raw_average_key_size": 28, "raw_value_size": 108261584, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927006, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290708, "oldest_key_time": 0, "file_creation_time": 1732290708, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234806, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:52.205365 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234807.sst) /dev/shm/ToplingTemp/Topling-339409-klqQlI 2024/11/22-23:51:52.939437 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234807.sst): first pass time = 0.73's, 182.850'MB/sec 2024/11/22-23:51:54.502792 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234807.sst): old prealloc_size = 147639500, real_size = 36408144 2024/11/22-23:51:54.503313 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234807.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926968 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.79 avg-zval = 37.14 2024/11/22-23:51:54.503354 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234807.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:55.143060 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234807.sst): second pass time = 1.56's, 69.281'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.205'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3485.619'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.450'MB/sec write SST data time = 0.12's, 298.252'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26684.828'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.281'MB/sec zip pipeline throughput = 69.281'MB/sec entries = 926968 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.79 avg-zval = 37.14 usrkeys = 926968 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.79 avg-zval = 37.14 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5826 value = 3.1449 dict = 1.00 all = 3.6016 } Zip/UnZip{ index = 0.0945 value = 0.3180 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 1.362847 GB avg = 0.123 KB (by entry num) total key len = 0.311021 GB avg = 0.028 KB total ukey len = 0.222158 GB avg = 0.020 KB total ukey num = 0.011107905 Billion total entry num = 0.011107905 Billion write speed all = 0.031013616 MB/sec (with seq num) write speed all = 0.029367148 MB/sec (without seq num) 2024/11/22-23:51:55.176089 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234807: 926968 keys, 37266300 bytes, temperature: kWarm 2024/11/22-23:51:55.176149 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290715176117, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234807, "file_size": 37266300, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34424320, "index_size": 1751872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955104, "raw_average_key_size": 28, "raw_value_size": 108262720, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926968, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290712, "oldest_key_time": 0, "file_creation_time": 1732290712, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234807, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:55.212409 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234808.sst) /dev/shm/ToplingTemp/Topling-339409-KIrg0G 2024/11/22-23:51:56.110034 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234808.sst): first pass time = 0.90's, 149.532'MB/sec 2024/11/22-23:51:57.449237 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234808.sst): old prealloc_size = 147639500, real_size = 32673344 2024/11/22-23:51:57.449745 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234808.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920183 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0307 GB avg-val = 117.86 avg-zval = 33.34 2024/11/22-23:51:57.450474 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234808.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:51:58.331055 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234808.sst): second pass time = 1.34's, 81.028'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.541'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3610.294'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.307'MB/sec write SST data time = 0.13's, 276.159'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 28397.255'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.028'MB/sec zip pipeline throughput = 81.028'MB/sec entries = 920183 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.86 avg-zval = 33.34 usrkeys = 920183 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.05 avg-zval = 33.34 TagRS{ kinds = RS_Key0_TagN bytes = 143880 } TagArray{ size = 813921 bytes = 2950480 } seq expand size = 5697447 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4414 value = 3.5349 dict = 1.00 all = 3.6640 } Zip/UnZip{ index = 0.0958 value = 0.2829 dict = 1.00 all = 0.2729 } ---------------------------- total value len = 1.476997 GB avg = 0.123 KB (by entry num) total key len = 0.336786 GB avg = 0.028 KB total ukey len = 0.240562 GB avg = 0.020 KB total ukey num = 0.012028088 Billion total entry num = 0.012028088 Billion write speed all = 0.033603996 MB/sec (with seq num) write speed all = 0.031821240 MB/sec (without seq num) 2024/11/22-23:51:58.364214 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234808: 920183 keys, 36631288 bytes, temperature: kWarm 2024/11/22-23:51:58.364275 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290718364245, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234808, "file_size": 36631288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3632592150, "table_properties": {"data_size": 30680560, "index_size": 1762560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25765124, "raw_average_key_size": 28, "raw_value_size": 108452670, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920183, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290715, "oldest_key_time": 0, "file_creation_time": 1732290715, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234808, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:51:58.407664 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234809.sst) /dev/shm/ToplingTemp/Topling-339409-M1NJHH 2024/11/22-23:51:59.340319 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234809.sst): first pass time = 0.93's, 143.917'MB/sec 2024/11/22-23:52:00.676819 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234809.sst): old prealloc_size = 147639500, real_size = 34053328 2024/11/22-23:52:00.677368 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234809.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930380 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0320 GB avg-val = 116.26 avg-zval = 34.44 2024/11/22-23:52:00.679084 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234809.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:52:01.594131 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234809.sst): second pass time = 1.34's, 80.972'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.602'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3344.117'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 295.545'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34372.431'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.972'MB/sec zip pipeline throughput = 80.972'MB/sec entries = 930380 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.26 avg-zval = 34.44 usrkeys = 930380 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.26 avg-zval = 34.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930380 bytes = 3256352 } seq expand size = 6512660 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4441 value = 3.3761 dict = 1.00 all = 3.5171 } Zip/UnZip{ index = 0.0957 value = 0.2962 dict = 1.00 all = 0.2843 } ---------------------------- total value len = 1.591677 GB avg = 0.123 KB (by entry num) total key len = 0.362837 GB avg = 0.028 KB total ukey len = 0.259169 GB avg = 0.020 KB total ukey num = 0.012958468 Billion total entry num = 0.012958468 Billion write speed all = 0.036209122 MB/sec (with seq num) write speed all = 0.034288584 MB/sec (without seq num) 2024/11/22-23:52:01.628118 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234809: 930380 keys, 38161606 bytes, temperature: kWarm 2024/11/22-23:52:01.628181 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290721628149, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234809, "file_size": 38161606, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3509327862, "largest_seqno": 3655386380, "table_properties": {"data_size": 32038912, "index_size": 1781632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050640, "raw_average_key_size": 28, "raw_value_size": 108167140, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930380, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290718, "oldest_key_time": 0, "file_creation_time": 1732290718, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234809, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:52:01.665013 140025205032512 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234810.sst) /dev/shm/ToplingTemp/Topling-339409-t0iX4G 2024/11/22-23:52:01.729307 140025205032512 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234810.sst): first pass time = 0.06's, 233.181'MB/sec 2024/11/22-23:52:01.861323 140025205032512 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234810.sst): old prealloc_size = 147639500, real_size = 4205008 2024/11/22-23:52:01.861428 140025205032512 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234810.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 105517 prefix = 4 raw-key = 0.0017 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0120 GB zip-val = 0.0040 GB avg-val = 114.00 avg-zval = 37.67 2024/11/22-23:52:01.861539 140025205032512 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234810.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.015 GB 2024/11/22-23:52:01.954365 140025205032512 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00/234810.sst): second pass time = 0.13's, 91.520'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.09's, 49.822'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2409.446'MB/sec (index lex order gen) rebuild zvType time = 0.08's, 0.337'MB/sec write SST data time = 0.01's, 315.627'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.125'MB, throughput = 4278.852'MB/sec zip my value time = 0.13's, unzip length = 0.012'GB zip my value throughput = 91.520'MB/sec zip pipeline throughput = 91.520'MB/sec entries = 105517 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.00 avg-zval = 37.67 usrkeys = 105517 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.00 avg-zval = 37.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 105517 bytes = 329760 } seq expand size = 738619 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0021 GB value = 0.0120 GB dict = 0.13 MB all = 0.0150 GB } __ZipSize{ index = 0.0002 GB value = 0.0040 GB dict = 0.13 MB all = 0.0046 GB } UnZip/Zip{ index = 10.3725 value = 3.0262 dict = 1.00 all = 3.2328 } Zip/UnZip{ index = 0.0964 value = 0.3305 dict = 1.00 all = 0.3093 } ---------------------------- total value len = 1.604444 GB avg = 0.123 KB (by entry num) total key len = 0.365792 GB avg = 0.028 KB total ukey len = 0.261280 GB avg = 0.020 KB total ukey num = 0.013063985 Billion total entry num = 0.013063985 Billion write speed all = 0.036500143 MB/sec (with seq num) write speed all = 0.034563980 MB/sec (without seq num) 2024/11/22-23:52:01.964992 140025205032512 [compaction_job.cc:1948] [default] [JOB 14454] Generated table #234810: 105517 keys, 4634849 bytes, temperature: kWarm 2024/11/22-23:52:01.965052 140025205032512 EVENT_LOG_v1 {"time_micros": 1732290721965021, "cf_name": "default", "job": 14454, "event": "table_file_creation", "file_number": 234810, "file_size": 4634849, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3627948246, "largest_seqno": 3647181046, "table_properties": {"data_size": 3974976, "index_size": 203456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2954476, "raw_average_key_size": 28, "raw_value_size": 12028938, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 105517, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732290721, "oldest_key_time": 0, "file_creation_time": 1732290721, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 234810, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:52:01.976998 140025205032512 [compaction_job.cc:703] job-14454: subcompact[0], size: 0.515986 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-14454/att-00 [234796,234797,234798,234799,234800,234801,234802,234803,234804,234805,234806,234807,234808,234809,234810] 2024/11/22-23:52:02.070181 140025205032512 (Original Log Time 2024/11/22-23:51:10.792243) EVENT_LOG_v1 {"time_micros": 1732290670792201, "job": 14454, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [234789], "files_L3": [234726, 234727, 234728, 234729, 234730, 234731, 234732, 234733, 234734, 234735, 234736, 234737, 234738, 234739], "score": 0.999105, "input_data_size": 531291230, "oldest_snapshot_seqno": 3655408668} 2024/11/22-23:52:02.070185 140025205032512 (Original Log Time 2024/11/22-23:52:02.068988) [compaction_job.cc:2022] [default] [JOB 14454] Compacted 1@2 + 14@3 files to L3 => 515986339 bytes 2024/11/22-23:52:02.070186 140025205032512 (Original Log Time 2024/11/22-23:52:02.070118) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.4 rd, 10.1 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(20.3, 486.4 +0.0 blob) out(492.1 +0.0 blob), read-write-amplify(49.2) write-amplify(24.2) OK, records in: 13135222, records dropped: 71237 output_compression: Snappy 2024/11/22-23:52:02.070193 140025205032512 (Original Log Time 2024/11/22-23:52:02.070161) EVENT_LOG_v1 {"time_micros": 1732290722070141, "cf": "default", "job": 14454, "event": "compaction_finished", "compaction_time_micros": 51184764, "compaction_time_cpu_micros": 23078282, "output_level": 3, "num_output_files": 15, "total_output_size": 515986339, "num_input_records": 13135222, "num_output_records": 13063985, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 354236790, "file_range_sync_nanos": 0, "file_fsync_nanos": 14604, "file_prepare_write_nanos": 54637, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/22-23:52:02.190374 140025205032512 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732290670,"executesMs":51431,"compactionJobId":14454,"attempt":0,"compactionInputRawBytes":1904148224,"compactionInputZipBytes":531291230,"compactionOutputRawBytes":0,"compactionOutputZipBytes":515986339,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5d6603d7ac1eb4f74c58f7f5e4cc084c-c566345712bc970b-00"}}, response = 2024/11/22-23:52:02.190447 140025205032512 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-14454/att-00: olev 3, work 51.432 s, result 4.926 ms, install 1.228 ms, input{raw 1.773 GiB zip 506.679 MiB}