2024/11/24-01:38:54.675467 139957659977280 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00: fork to child time = 0.011174 sec 2024/11/24-01:38:54.676225 139957659977280 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00 2024/11/24-01:38:54.682856 139957659977280 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00 2024/11/24-01:38:54.683830 139957659977280 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-01:38:54.684126 139957659977280 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-01:38:54.746752 139957659977280 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39466/att-00/MANIFEST-206347 2024/11/24-01:38:54.762535 139957659977280 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39466/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 287516, last_sequence is 4376728749, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 287389 2024/11/24-01:38:54.762542 139957659977280 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-01:38:54.762647 139957659977280 [version_set.cc:5721] Creating manifest 287516 2024/11/24-01:38:54.820604 139957659977280 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-01:38:54.844533 139957659977280 [compaction_job.cc:2437] [default] [JOB 39466] Compacting 1@2 + 14@3 files to L3, score 0.72, subcompactions 1 : 1 2024/11/24-01:38:54.844553 139957659977280 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 29570 Base level 2, inputs: [287512(27MB)], [287450(33MB) 287451(33MB) 287452(33MB) 287453(33MB) 287454(33MB) 287455(34MB) 287456(34MB) 287457(34MB) 287458(31MB) 287459(35MB) 287460(31MB) 287461(34MB) 287462(34MB) 287469(32MB)] 2024/11/24-01:38:54.851421 139957659977280 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:38:54.851455 139957659977280 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:38:54.851475 139957659977280 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:38:54.851481 139957659977280 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:38:54.851492 139957659977280 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:38:55.442651 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287517.sst) /dev/shm/ToplingTemp/Topling-239332-NgjdRk 2024/11/24-01:38:56.004437 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287517.sst): first pass time = 0.56's, 238.927'MB/sec 2024/11/24-01:38:57.488524 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287517.sst): old prealloc_size = 147639500, real_size = 31255216 2024/11/24-01:38:57.489123 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287517.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925881 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0293 GB avg-val = 116.96 avg-zval = 31.60 2024/11/24-01:38:57.489903 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287517.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:38:58.312383 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287517.sst): second pass time = 1.48's, 73.008'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.769'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3070.185'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 265.734'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37249.259'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.008'MB/sec zip pipeline throughput = 73.344'MB/sec entries = 925881 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 31.60 usrkeys = 925881 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 31.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925881 bytes = 3124864 } seq expand size = 6481167 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.5107 value = 3.7008 dict = 1.00 all = 3.8092 } Zip/UnZip{ index = 0.0951 value = 0.2702 dict = 1.00 all = 0.2625 } ---------------------------- total value len = 8.963394 GB avg = 0.168 KB (by entry num) total key len = 1.466746 GB avg = 0.028 KB total ukey len = 1.016396 GB avg = 0.015 KB total ukey num = 0.066476316 Billion total entry num = 0.053284045 Billion write speed all = 0.173183058 MB/sec (with seq num) write speed all = 0.166105190 MB/sec (without seq num) 2024/11/24-01:38:58.359632 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287517: 925881 keys, 35235262 bytes, temperature: kWarm 2024/11/24-01:38:58.359721 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383538359666, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287517, "file_size": 35235262, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4376713949, "table_properties": {"data_size": 29261792, "index_size": 1761792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924668, "raw_average_key_size": 28, "raw_value_size": 108293078, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925881, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383535, "oldest_key_time": 0, "file_creation_time": 1732383535, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287517, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:38:58.400203 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287518.sst) /dev/shm/ToplingTemp/Topling-239332-P7VPGn 2024/11/24-01:38:59.687363 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287518.sst): first pass time = 1.29's, 104.278'MB/sec 2024/11/24-01:39:02.974569 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287518.sst): old prealloc_size = 147639500, real_size = 31108944 2024/11/24-01:39:02.975149 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287518.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925923 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 116.96 avg-zval = 31.43 2024/11/24-01:39:02.975936 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287518.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:03.908725 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287518.sst): second pass time = 3.29's, 32.951'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 37.563'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3149.457'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 273.256'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34607.295'MB/sec zip my value time = 3.29's, unzip length = 0.108'GB zip my value throughput = 32.951'MB/sec zip pipeline throughput = 112.394'MB/sec entries = 925923 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.96 avg-zval = 31.43 usrkeys = 925923 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.96 avg-zval = 31.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925923 bytes = 3125008 } seq expand size = 6481461 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4432 value = 3.7209 dict = 1.00 all = 3.8251 } Zip/UnZip{ index = 0.0958 value = 0.2688 dict = 1.00 all = 0.2614 } ---------------------------- total value len = 9.078168 GB avg = 0.167 KB (by entry num) total key len = 1.492672 GB avg = 0.028 KB total ukey len = 1.034914 GB avg = 0.015 KB total ukey num = 0.067402239 Billion total entry num = 0.054209968 Billion write speed all = 0.175502935 MB/sec (with seq num) write speed all = 0.168302743 MB/sec (without seq num) 2024/11/24-01:39:03.953405 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287518: 925923 keys, 35088954 bytes, temperature: kWarm 2024/11/24-01:39:03.953469 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383543953435, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287518, "file_size": 35088954, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4376713755, "table_properties": {"data_size": 29104000, "index_size": 1773248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25925844, "raw_average_key_size": 28, "raw_value_size": 108292010, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925923, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383538, "oldest_key_time": 0, "file_creation_time": 1732383538, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287518, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:03.989544 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287519.sst) /dev/shm/ToplingTemp/Topling-239332-IacrIn 2024/11/24-01:39:04.874259 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287519.sst): first pass time = 0.88's, 151.713'MB/sec 2024/11/24-01:39:07.365531 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287519.sst): old prealloc_size = 147639500, real_size = 31135840 2024/11/24-01:39:07.366240 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287519.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925901 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 116.96 avg-zval = 31.48 2024/11/24-01:39:07.367086 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287519.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:08.290388 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287519.sst): second pass time = 2.49's, 43.485'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 37.969'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2556.759'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.302'MB/sec write SST data time = 0.16's, 223.128'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35290.165'MB/sec zip my value time = 2.49's, unzip length = 0.108'GB zip my value throughput = 43.485'MB/sec zip pipeline throughput = 76.442'MB/sec entries = 925901 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 31.48 usrkeys = 925901 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 31.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925901 bytes = 3124928 } seq expand size = 6481307 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5140 value = 3.7159 dict = 1.00 all = 3.8221 } Zip/UnZip{ index = 0.0951 value = 0.2691 dict = 1.00 all = 0.2616 } ---------------------------- total value len = 9.192941 GB avg = 0.167 KB (by entry num) total key len = 1.518597 GB avg = 0.028 KB total ukey len = 1.053432 GB avg = 0.015 KB total ukey num = 0.068328140 Billion total entry num = 0.055135869 Billion write speed all = 0.177825964 MB/sec (with seq num) write speed all = 0.170503326 MB/sec (without seq num) 2024/11/24-01:39:08.334362 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287519: 925901 keys, 35115878 bytes, temperature: kWarm 2024/11/24-01:39:08.334431 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383548334396, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287519, "file_size": 35115878, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4376713906, "table_properties": {"data_size": 29142928, "index_size": 1761280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25925228, "raw_average_key_size": 28, "raw_value_size": 108292606, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925901, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383543, "oldest_key_time": 0, "file_creation_time": 1732383543, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287519, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:08.380840 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287520.sst) /dev/shm/ToplingTemp/Topling-239332-lu1fJm 2024/11/24-01:39:09.390822 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287520.sst): first pass time = 1.01's, 132.895'MB/sec 2024/11/24-01:39:10.786772 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287520.sst): old prealloc_size = 147639500, real_size = 30902448 2024/11/24-01:39:10.787507 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287520.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926181 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 116.92 avg-zval = 31.21 2024/11/24-01:39:10.788323 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287520.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:12.015778 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287520.sst): second pass time = 1.40's, 77.617'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.23's, 28.384'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2472.649'MB/sec (index lex order gen) rebuild zvType time = 1.03's, 0.225'MB/sec write SST data time = 0.20's, 175.028'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35404.017'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.617'MB/sec zip pipeline throughput = 77.617'MB/sec entries = 926181 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.92 avg-zval = 31.21 usrkeys = 926181 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.92 avg-zval = 31.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926181 bytes = 3125872 } seq expand size = 6483267 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.5110 value = 3.7458 dict = 1.00 all = 3.8476 } Zip/UnZip{ index = 0.0951 value = 0.2670 dict = 1.00 all = 0.2599 } ---------------------------- total value len = 9.307709 GB avg = 0.166 KB (by entry num) total key len = 1.544530 GB avg = 0.028 KB total ukey len = 1.071956 GB avg = 0.015 KB total ukey num = 0.069254321 Billion total entry num = 0.056062050 Billion write speed all = 0.180150649 MB/sec (with seq num) write speed all = 0.172705465 MB/sec (without seq num) 2024/11/24-01:39:12.079859 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287520: 926181 keys, 34883142 bytes, temperature: kWarm 2024/11/24-01:39:12.079931 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383552079887, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287520, "file_size": 34883142, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4376713553, "table_properties": {"data_size": 28908384, "index_size": 1762304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933068, "raw_average_key_size": 28, "raw_value_size": 108284682, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926181, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383548, "oldest_key_time": 0, "file_creation_time": 1732383548, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287520, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:12.125175 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287521.sst) /dev/shm/ToplingTemp/Topling-239332-HlGW8l 2024/11/24-01:39:13.357254 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287521.sst): first pass time = 1.23's, 108.939'MB/sec 2024/11/24-01:39:14.881780 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287521.sst): old prealloc_size = 147639500, real_size = 31038240 2024/11/24-01:39:14.882508 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287521.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925936 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 116.95 avg-zval = 31.37 2024/11/24-01:39:14.883330 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287521.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:15.848062 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287521.sst): second pass time = 1.52's, 71.103'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 36.241'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2502.634'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.15's, 226.115'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34981.160'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.103'MB/sec zip pipeline throughput = 71.103'MB/sec entries = 925936 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 31.37 usrkeys = 925936 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 31.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925936 bytes = 3125056 } seq expand size = 6481552 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5147 value = 3.7284 dict = 1.00 all = 3.8328 } Zip/UnZip{ index = 0.0951 value = 0.2682 dict = 1.00 all = 0.2609 } ---------------------------- total value len = 9.422483 GB avg = 0.165 KB (by entry num) total key len = 1.570456 GB avg = 0.028 KB total ukey len = 1.090474 GB avg = 0.016 KB total ukey num = 0.070180257 Billion total entry num = 0.056987986 Billion write speed all = 0.182474695 MB/sec (with seq num) write speed all = 0.174907026 MB/sec (without seq num) 2024/11/24-01:39:15.892851 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287521: 925936 keys, 35018254 bytes, temperature: kWarm 2024/11/24-01:39:15.892931 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383555892888, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287521, "file_size": 35018254, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4262912326, "largest_seqno": 4376712874, "table_properties": {"data_size": 29045328, "index_size": 1761216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926208, "raw_average_key_size": 28, "raw_value_size": 108291596, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925936, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383552, "oldest_key_time": 0, "file_creation_time": 1732383552, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287521, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:15.936099 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287522.sst) /dev/shm/ToplingTemp/Topling-239332-xD7s0l 2024/11/24-01:39:16.934578 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287522.sst): first pass time = 1.00's, 134.427'MB/sec 2024/11/24-01:39:18.290474 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287522.sst): old prealloc_size = 147639500, real_size = 31197056 2024/11/24-01:39:18.291054 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287522.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925978 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0292 GB avg-val = 116.95 avg-zval = 31.54 2024/11/24-01:39:18.291779 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287522.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:19.212567 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287522.sst): second pass time = 1.36's, 79.913'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 38.150'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3160.189'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 269.765'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33854.771'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.913'MB/sec zip pipeline throughput = 79.913'MB/sec entries = 925978 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 31.54 usrkeys = 925978 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 31.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925978 bytes = 3125184 } seq expand size = 6481846 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.5129 value = 3.7081 dict = 1.00 all = 3.8155 } Zip/UnZip{ index = 0.0951 value = 0.2697 dict = 1.00 all = 0.2621 } ---------------------------- total value len = 9.537255 GB avg = 0.165 KB (by entry num) total key len = 1.596383 GB avg = 0.028 KB total ukey len = 1.108994 GB avg = 0.016 KB total ukey num = 0.071106235 Billion total entry num = 0.057913964 Billion write speed all = 0.184799884 MB/sec (with seq num) write speed all = 0.177109680 MB/sec (without seq num) 2024/11/24-01:39:19.259532 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287522: 925978 keys, 35177304 bytes, temperature: kWarm 2024/11/24-01:39:19.259600 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383559259567, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287522, "file_size": 35177304, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4376712819, "table_properties": {"data_size": 29203760, "index_size": 1761600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927384, "raw_average_key_size": 28, "raw_value_size": 108290400, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925978, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383555, "oldest_key_time": 0, "file_creation_time": 1732383555, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287522, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:19.314234 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287523.sst) /dev/shm/ToplingTemp/Topling-239332-X96sin 2024/11/24-01:39:20.319496 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287523.sst): first pass time = 1.01's, 133.520'MB/sec 2024/11/24-01:39:21.682924 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287523.sst): old prealloc_size = 147639500, real_size = 31450112 2024/11/24-01:39:21.683482 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287523.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925807 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0295 GB avg-val = 116.97 avg-zval = 31.82 2024/11/24-01:39:21.684229 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287523.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:22.500932 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287523.sst): second pass time = 1.36's, 79.503'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.313'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3264.753'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 275.736'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36240.172'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.503'MB/sec zip pipeline throughput = 79.503'MB/sec entries = 925807 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 31.82 usrkeys = 925807 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 31.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925807 bytes = 3124608 } seq expand size = 6480649 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5125 value = 3.6764 dict = 1.00 all = 3.7883 } Zip/UnZip{ index = 0.0951 value = 0.2720 dict = 1.00 all = 0.2640 } ---------------------------- total value len = 9.652031 GB avg = 0.164 KB (by entry num) total key len = 1.622306 GB avg = 0.028 KB total ukey len = 1.127510 GB avg = 0.016 KB total ukey num = 0.072032042 Billion total entry num = 0.058839771 Billion write speed all = 0.187125031 MB/sec (with seq num) write speed all = 0.179312319 MB/sec (without seq num) 2024/11/24-01:39:22.545135 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287523: 925807 keys, 35429528 bytes, temperature: kWarm 2024/11/24-01:39:22.545204 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383562545168, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287523, "file_size": 35429528, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4376713343, "table_properties": {"data_size": 29457136, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25922596, "raw_average_key_size": 28, "raw_value_size": 108295206, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925807, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383559, "oldest_key_time": 0, "file_creation_time": 1732383559, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287523, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:22.586224 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287524.sst) /dev/shm/ToplingTemp/Topling-239332-rHrHIo 2024/11/24-01:39:23.482613 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287524.sst): first pass time = 0.90's, 149.737'MB/sec 2024/11/24-01:39:24.816962 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287524.sst): old prealloc_size = 147639500, real_size = 31721184 2024/11/24-01:39:24.817522 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287524.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926041 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 116.94 avg-zval = 32.09 2024/11/24-01:39:24.818264 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287524.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:25.633718 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287524.sst): second pass time = 1.33's, 81.203'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.717'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3440.720'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 283.252'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35901.993'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.203'MB/sec zip pipeline throughput = 81.203'MB/sec entries = 926041 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.94 avg-zval = 32.09 usrkeys = 926041 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.94 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926041 bytes = 3125408 } seq expand size = 6482287 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4453 value = 3.6441 dict = 1.00 all = 3.7590 } Zip/UnZip{ index = 0.0957 value = 0.2744 dict = 1.00 all = 0.2660 } ---------------------------- total value len = 9.766802 GB avg = 0.163 KB (by entry num) total key len = 1.648235 GB avg = 0.028 KB total ukey len = 1.146031 GB avg = 0.016 KB total ukey num = 0.072958083 Billion total entry num = 0.059765812 Billion write speed all = 0.189450441 MB/sec (with seq num) write speed all = 0.181515181 MB/sec (without seq num) 2024/11/24-01:39:25.679525 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287524: 926041 keys, 35705306 bytes, temperature: kWarm 2024/11/24-01:39:25.679593 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383565679558, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287524, "file_size": 35705306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4376713832, "table_properties": {"data_size": 29716368, "index_size": 1773120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25929148, "raw_average_key_size": 28, "raw_value_size": 108288658, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926041, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383562, "oldest_key_time": 0, "file_creation_time": 1732383562, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287524, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:25.719368 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287525.sst) /dev/shm/ToplingTemp/Topling-239332-rpCNrk 2024/11/24-01:39:26.549503 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287525.sst): first pass time = 0.83's, 161.689'MB/sec 2024/11/24-01:39:27.842866 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287525.sst): old prealloc_size = 147639500, real_size = 29591952 2024/11/24-01:39:27.843426 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287525.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935101 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0276 GB avg-val = 115.53 avg-zval = 29.48 2024/11/24-01:39:27.844787 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287525.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:28.663344 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287525.sst): second pass time = 1.29's, 83.607'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 40.948'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3399.313'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 263.658'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31246.617'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.607'MB/sec zip pipeline throughput = 83.607'MB/sec entries = 935101 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.53 avg-zval = 29.48 usrkeys = 935101 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.53 avg-zval = 29.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935101 bytes = 3155984 } seq expand size = 6545707 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0276 GB dict = 1.08 MB all = 0.0336 GB } UnZip/Zip{ index = 10.4442 value = 3.9189 dict = 1.00 all = 3.9950 } Zip/UnZip{ index = 0.0957 value = 0.2552 dict = 1.00 all = 0.2503 } ---------------------------- total value len = 9.881382 GB avg = 0.163 KB (by entry num) total key len = 1.674418 GB avg = 0.028 KB total ukey len = 1.164733 GB avg = 0.016 KB total ukey num = 0.073893184 Billion total entry num = 0.060700913 Billion write speed all = 0.191776989 MB/sec (with seq num) write speed all = 0.183717980 MB/sec (without seq num) 2024/11/24-01:39:28.705879 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287525: 935101 keys, 33596512 bytes, temperature: kWarm 2024/11/24-01:39:28.705959 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383568705923, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287525, "file_size": 33596512, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4376712948, "table_properties": {"data_size": 27567360, "index_size": 1790656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26182828, "raw_average_key_size": 28, "raw_value_size": 108034998, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935101, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383565, "oldest_key_time": 0, "file_creation_time": 1732383565, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287525, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:28.743566 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287526.sst) /dev/shm/ToplingTemp/Topling-239332-YBaMRn 2024/11/24-01:39:29.753190 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287526.sst): first pass time = 1.01's, 132.944'MB/sec 2024/11/24-01:39:31.069773 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287526.sst): old prealloc_size = 147639500, real_size = 32778384 2024/11/24-01:39:31.070365 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287526.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932184 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 115.98 avg-zval = 33.01 2024/11/24-01:39:31.071129 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287526.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:31.931439 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287526.sst): second pass time = 1.32's, 82.212'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.680'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3086.667'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 288.266'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35918.288'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.212'MB/sec zip pipeline throughput = 82.212'MB/sec entries = 932184 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.98 avg-zval = 33.01 usrkeys = 932184 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.98 avg-zval = 33.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932184 bytes = 3146128 } seq expand size = 6525288 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5120 value = 3.5135 dict = 1.00 all = 3.6497 } Zip/UnZip{ index = 0.0951 value = 0.2846 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 9.996024 GB avg = 0.162 KB (by entry num) total key len = 1.700519 GB avg = 0.028 KB total ukey len = 1.183377 GB avg = 0.016 KB total ukey num = 0.074825368 Billion total entry num = 0.061633097 Billion write speed all = 0.194102196 MB/sec (with seq num) write speed all = 0.185919868 MB/sec (without seq num) 2024/11/24-01:39:31.976860 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287526: 932184 keys, 36775132 bytes, temperature: kWarm 2024/11/24-01:39:31.976946 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383571976904, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287526, "file_size": 36775132, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4376713581, "table_properties": {"data_size": 30771584, "index_size": 1773568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26101152, "raw_average_key_size": 28, "raw_value_size": 108116624, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932184, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383568, "oldest_key_time": 0, "file_creation_time": 1732383568, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287526, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:32.017560 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287527.sst) /dev/shm/ToplingTemp/Topling-239332-MxDZVk 2024/11/24-01:39:32.969595 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287527.sst): first pass time = 0.95's, 140.986'MB/sec 2024/11/24-01:39:34.257958 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287527.sst): old prealloc_size = 147639500, real_size = 29113520 2024/11/24-01:39:34.258492 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287527.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 937897 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0271 GB avg-val = 115.11 avg-zval = 28.88 2024/11/24-01:39:34.260435 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287527.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:35.081929 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287527.sst): second pass time = 1.29's, 83.873'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 40.204'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3598.931'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.337'MB/sec write SST data time = 0.13's, 258.933'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 26369.716'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.873'MB/sec zip pipeline throughput = 83.873'MB/sec entries = 937897 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.11 avg-zval = 28.88 usrkeys = 937897 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.11 avg-zval = 28.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 937897 bytes = 3165424 } seq expand size = 6565279 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0271 GB dict = 1.08 MB all = 0.0331 GB } UnZip/Zip{ index = 10.4441 value = 3.9862 dict = 1.00 all = 4.0518 } Zip/UnZip{ index = 0.0957 value = 0.2509 dict = 1.00 all = 0.2468 } ---------------------------- total value len = 10.110546 GB avg = 0.162 KB (by entry num) total key len = 1.726780 GB avg = 0.028 KB total ukey len = 1.202135 GB avg = 0.016 KB total ukey num = 0.075763265 Billion total entry num = 0.062570994 Billion write speed all = 0.196428198 MB/sec (with seq num) write speed all = 0.188121790 MB/sec (without seq num) 2024/11/24-01:39:35.123169 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287527: 937897 keys, 33125810 bytes, temperature: kWarm 2024/11/24-01:39:35.123238 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383575123202, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287527, "file_size": 33125810, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4283369801, "largest_seqno": 4376712959, "table_properties": {"data_size": 27082848, "index_size": 1796032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26261116, "raw_average_key_size": 28, "raw_value_size": 107956662, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 937897, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383572, "oldest_key_time": 0, "file_creation_time": 1732383572, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287527, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:35.165806 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287528.sst) /dev/shm/ToplingTemp/Topling-239332-oJeC3n 2024/11/24-01:39:35.983913 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287528.sst): first pass time = 0.82's, 164.066'MB/sec 2024/11/24-01:39:37.313523 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287528.sst): old prealloc_size = 147639500, real_size = 32948896 2024/11/24-01:39:37.314119 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287528.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924675 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.15 avg-zval = 33.48 2024/11/24-01:39:37.314853 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287528.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:38.157272 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287528.sst): second pass time = 1.33's, 81.521'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.765'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3055.795'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 295.769'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34927.009'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.521'MB/sec zip pipeline throughput = 81.521'MB/sec entries = 924675 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 33.48 usrkeys = 924675 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 33.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924675 bytes = 3120800 } seq expand size = 6472725 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5149 value = 3.4991 dict = 1.00 all = 3.6349 } Zip/UnZip{ index = 0.0951 value = 0.2858 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 10.225346 GB avg = 0.161 KB (by entry num) total key len = 1.752671 GB avg = 0.028 KB total ukey len = 1.220628 GB avg = 0.016 KB total ukey num = 0.076687940 Billion total entry num = 0.063495669 Billion write speed all = 0.198752670 MB/sec (with seq num) write speed all = 0.190323940 MB/sec (without seq num) 2024/11/24-01:39:38.203543 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287528: 924675 keys, 36925260 bytes, temperature: kWarm 2024/11/24-01:39:38.203613 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383578203578, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287528, "file_size": 36925260, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4376713806, "table_properties": {"data_size": 30958736, "index_size": 1758784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25890900, "raw_average_key_size": 28, "raw_value_size": 108326926, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924675, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383575, "oldest_key_time": 0, "file_creation_time": 1732383575, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287528, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:38.239461 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287529.sst) /dev/shm/ToplingTemp/Topling-239332-0oP59l 2024/11/24-01:39:39.216047 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287529.sst): first pass time = 0.98's, 137.441'MB/sec 2024/11/24-01:39:40.552697 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287529.sst): old prealloc_size = 147639500, real_size = 32252512 2024/11/24-01:39:40.553337 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287529.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932990 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 115.86 avg-zval = 32.40 2024/11/24-01:39:40.554147 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287529.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:41.434442 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287529.sst): second pass time = 1.34's, 80.914'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.115'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2899.502'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 285.829'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33435.571'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.914'MB/sec zip pipeline throughput = 80.914'MB/sec entries = 932990 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.86 avg-zval = 32.40 usrkeys = 932990 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.86 avg-zval = 32.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932990 bytes = 3148864 } seq expand size = 6530930 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4442 value = 3.5754 dict = 1.00 all = 3.7023 } Zip/UnZip{ index = 0.0957 value = 0.2797 dict = 1.00 all = 0.2701 } ---------------------------- total value len = 10.339971 GB avg = 0.160 KB (by entry num) total key len = 1.778795 GB avg = 0.028 KB total ukey len = 1.239288 GB avg = 0.016 KB total ukey num = 0.077620930 Billion total entry num = 0.064428659 Billion write speed all = 0.201077195 MB/sec (with seq num) write speed all = 0.192525080 MB/sec (without seq num) 2024/11/24-01:39:41.478930 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287529: 932990 keys, 36252098 bytes, temperature: kWarm 2024/11/24-01:39:41.479010 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383581478966, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287529, "file_size": 36252098, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4376713136, "table_properties": {"data_size": 30232464, "index_size": 1786624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26123720, "raw_average_key_size": 28, "raw_value_size": 108094016, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932990, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383578, "oldest_key_time": 0, "file_creation_time": 1732383578, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287529, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:41.517731 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287530.sst) /dev/shm/ToplingTemp/Topling-239332-aJZ3Rk 2024/11/24-01:39:42.398085 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287530.sst): first pass time = 0.88's, 152.465'MB/sec 2024/11/24-01:39:43.723756 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287530.sst): old prealloc_size = 147639500, real_size = 30658496 2024/11/24-01:39:43.724359 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287530.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926655 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0287 GB avg-val = 116.84 avg-zval = 30.92 2024/11/24-01:39:43.725143 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287530.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:39:44.535513 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287530.sst): second pass time = 1.32's, 81.766'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.673'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3064.196'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.339'MB/sec write SST data time = 0.13's, 271.990'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34103.572'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.766'MB/sec zip pipeline throughput = 81.766'MB/sec entries = 926655 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.84 avg-zval = 30.92 usrkeys = 926655 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.84 avg-zval = 30.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926655 bytes = 3127472 } seq expand size = 6486585 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4440 value = 3.7788 dict = 1.00 all = 3.8748 } Zip/UnZip{ index = 0.0957 value = 0.2646 dict = 1.00 all = 0.2581 } ---------------------------- total value len = 10.454729 GB avg = 0.160 KB (by entry num) total key len = 1.804741 GB avg = 0.028 KB total ukey len = 1.257821 GB avg = 0.016 KB total ukey num = 0.078547585 Billion total entry num = 0.065355314 Billion write speed all = 0.203401326 MB/sec (with seq num) write speed all = 0.194726656 MB/sec (without seq num) 2024/11/24-01:39:44.578556 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287530: 926655 keys, 34638962 bytes, temperature: kWarm 2024/11/24-01:39:44.578635 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383584578592, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287530, "file_size": 34638962, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4297680095, "largest_seqno": 4376712980, "table_properties": {"data_size": 28652144, "index_size": 1774528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946340, "raw_average_key_size": 28, "raw_value_size": 108271402, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926655, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383581, "oldest_key_time": 0, "file_creation_time": 1732383581, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287530, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:44.637602 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287531.sst) /dev/shm/ToplingTemp/Topling-239332-WOja3n 2024/11/24-01:39:44.718856 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287531.sst): first pass time = 0.08's, 233.129'MB/sec 2024/11/24-01:39:44.890586 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287531.sst): old prealloc_size = 147639500, real_size = 5323648 2024/11/24-01:39:44.890711 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287531.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 132922 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0152 GB zip-val = 0.0050 GB avg-val = 114.45 avg-zval = 37.87 2024/11/24-01:39:44.890858 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287531.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.019 GB 2024/11/24-01:39:45.006151 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00/287531.sst): second pass time = 0.17's, 88.953'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 51.059'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2491.070'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.342'MB/sec write SST data time = 0.02's, 320.683'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.159'MB, throughput = 5852.774'MB/sec zip my value time = 0.17's, unzip length = 0.015'GB zip my value throughput = 88.953'MB/sec zip pipeline throughput = 88.953'MB/sec entries = 132922 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.45 avg-zval = 37.87 usrkeys = 132922 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.45 avg-zval = 37.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 132922 bytes = 448624 } seq expand size = 930454 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0027 GB value = 0.0152 GB dict = 0.16 MB all = 0.0189 GB } __ZipSize{ index = 0.0003 GB value = 0.0050 GB dict = 0.16 MB all = 0.0059 GB } UnZip/Zip{ index = 10.3923 value = 3.0217 dict = 1.00 all = 3.2094 } Zip/UnZip{ index = 0.0962 value = 0.3309 dict = 1.00 all = 0.3116 } ---------------------------- total value len = 10.470872 GB avg = 0.160 KB (by entry num) total key len = 1.808463 GB avg = 0.028 KB total ukey len = 1.260479 GB avg = 0.016 KB total ukey num = 0.078680507 Billion total entry num = 0.065488236 Billion write speed all = 0.203729321 MB/sec (with seq num) write speed all = 0.195037076 MB/sec (without seq num) 2024/11/24-01:39:45.019537 139957659977280 [compaction_job.cc:1948] [default] [JOB 39466] Generated table #287531: 132922 keys, 5899735 bytes, temperature: kWarm 2024/11/24-01:39:45.019590 139957659977280 EVENT_LOG_v1 {"time_micros": 1732383585019562, "cf_name": "default", "job": 39466, "event": "table_file_creation", "file_number": 287531, "file_size": 5899735, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4375255128, "table_properties": {"data_size": 5034416, "index_size": 255808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3721816, "raw_average_key_size": 28, "raw_value_size": 15212644, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 132922, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383584, "oldest_key_time": 0, "file_creation_time": 1732383584, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287531, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:39:45.034168 139957659977280 [compaction_job.cc:703] job-39466: subcompact[0], size: 0.498867 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39466/att-00 [287517,287518,287519,287520,287521,287522,287523,287524,287525,287526,287527,287528,287529,287530,287531] 2024/11/24-01:39:45.122674 139957659977280 (Original Log Time 2024/11/24-01:38:54.844604) EVENT_LOG_v1 {"time_micros": 1732383534844566, "job": 39466, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [287512], "files_L3": [287450, 287451, 287452, 287453, 287454, 287455, 287456, 287457, 287458, 287459, 287460, 287461, 287462, 287469], "score": 0.715606, "input_data_size": 524520852, "oldest_snapshot_seqno": 4376728377} 2024/11/24-01:39:45.122678 139957659977280 (Original Log Time 2024/11/24-01:39:45.121229) [compaction_job.cc:2022] [default] [JOB 39466] Compacted 1@2 + 14@3 files to L3 => 498867137 bytes 2024/11/24-01:39:45.122679 139957659977280 (Original Log Time 2024/11/24-01:39:45.122610) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.5 rd, 9.9 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.1, 473.1 +0.0 blob) out(475.8 +0.0 blob), read-write-amplify(36.0) write-amplify(17.5) OK, records in: 13233024, records dropped: 102952 output_compression: Snappy 2024/11/24-01:39:45.122685 139957659977280 (Original Log Time 2024/11/24-01:39:45.122653) EVENT_LOG_v1 {"time_micros": 1732383585122633, "cf": "default", "job": 39466, "event": "compaction_finished", "compaction_time_micros": 50189569, "compaction_time_cpu_micros": 20837990, "output_level": 3, "num_output_files": 15, "total_output_size": 498867137, "num_input_records": 13233024, "num_output_records": 13130072, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 344940203, "file_range_sync_nanos": 0, "file_fsync_nanos": 9530, "file_prepare_write_nanos": 39413, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/24-01:39:45.237920 139957659977280 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732383534,"executesMs":50437,"compactionJobId":39466,"attempt":0,"compactionInputRawBytes":1912602708,"compactionInputZipBytes":524520852,"compactionOutputRawBytes":0,"compactionOutputZipBytes":498867137,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-99f0563bf0bbe4f021c3e28ce01908f1-06b5863294b16c46-00"}}, response = 2024/11/24-01:39:45.238036 139957659977280 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39466/att-00: olev 3, work 50.438 s, result 7.721 ms, install 1.480 ms, input{raw 1.781 GiB zip 500.222 MiB}