2024/11/23-18:47:07.134291 139957257127488 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00: fork to child time = 0.007960 sec 2024/11/23-18:47:07.134910 139957257127488 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00 2024/11/23-18:47:07.141700 139957257127488 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00 2024/11/23-18:47:07.142572 139957257127488 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-18:47:07.142781 139957257127488 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-18:47:07.193803 139957257127488 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33460/att-00/MANIFEST-206347 2024/11/23-18:47:07.224568 139957257127488 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33460/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 275636, last_sequence is 4194090210, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 275476 2024/11/23-18:47:07.224574 139957257127488 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-18:47:07.224861 139957257127488 [version_set.cc:5721] Creating manifest 275636 2024/11/23-18:47:07.258529 139957257127488 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-18:47:07.277338 139957257127488 [compaction_job.cc:2437] [default] [JOB 33460] Compacting 1@2 + 20@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-18:47:07.277355 139957257127488 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 25055 Base level 2, inputs: [275590(27MB)], [275610(36MB) 275611(3511KB) 275541(38MB) 275542(37MB) 275543(35MB) 275544(38MB) 275545(35MB) 275546(35MB) 275547(33MB) 275548(35MB) 275549(36MB) 275550(36MB) 275551(35MB) 275552(35MB) 275553(36MB) 275554(37MB) 275555(36MB) 275556(7220KB) 275499(37MB) 275500(37MB)] 2024/11/23-18:47:07.284028 139957257127488 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:47:07.284062 139957257127488 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:47:07.284083 139957257127488 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:47:07.284088 139957257127488 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:47:07.284102 139957257127488 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:47:07.980417 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275637.sst) /dev/shm/ToplingTemp/Topling-155195-NfoFnj 2024/11/23-18:47:08.523021 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275637.sst): first pass time = 0.54's, 247.378'MB/sec 2024/11/23-18:47:10.298723 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275637.sst): old prealloc_size = 147639500, real_size = 33788736 2024/11/23-18:47:10.299283 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275637.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925816 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0318 GB avg-val = 116.97 avg-zval = 34.34 2024/11/23-18:47:10.300060 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275637.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:11.305755 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275637.sst): second pass time = 1.77's, 61.014'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 37.506'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3288.718'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.23's, 165.384'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35357.110'MB/sec zip my value time = 1.77's, unzip length = 0.108'GB zip my value throughput = 61.014'MB/sec zip pipeline throughput = 120.216'MB/sec entries = 925816 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 34.34 usrkeys = 925816 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 34.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925816 bytes = 3124640 } seq expand size = 6480712 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5164 value = 3.4059 dict = 1.00 all = 3.5537 } Zip/UnZip{ index = 0.0951 value = 0.2936 dict = 1.00 all = 0.2814 } ---------------------------- total value len = 8.963395 GB avg = 0.168 KB (by entry num) total key len = 1.466744 GB avg = 0.028 KB total ukey len = 1.016394 GB avg = 0.015 KB total ukey num = 0.066476251 Billion total entry num = 0.053283980 Billion write speed all = 0.293648825 MB/sec (with seq num) write speed all = 0.281647622 MB/sec (without seq num) 2024/11/23-18:47:11.354873 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275637: 925816 keys, 37768344 bytes, temperature: kWarm 2024/11/23-18:47:11.355012 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358831354920, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275637, "file_size": 37768344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4122762187, "largest_seqno": 4193643639, "table_properties": {"data_size": 31796400, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25922848, "raw_average_key_size": 28, "raw_value_size": 108294932, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925816, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358827, "oldest_key_time": 0, "file_creation_time": 1732358827, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275637, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:11.399548 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275638.sst) /dev/shm/ToplingTemp/Topling-155195-5LJ7pl 2024/11/23-18:47:12.450041 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275638.sst): first pass time = 1.05's, 127.771'MB/sec 2024/11/23-18:47:13.776255 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275638.sst): old prealloc_size = 147639500, real_size = 34741920 2024/11/23-18:47:13.776835 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275638.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927827 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0327 GB avg-val = 116.66 avg-zval = 35.29 2024/11/23-18:47:13.777642 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275638.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:14.617292 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275638.sst): second pass time = 1.33's, 81.664'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.329'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3243.095'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 304.180'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34338.653'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.664'MB/sec zip pipeline throughput = 81.664'MB/sec entries = 927827 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 35.29 usrkeys = 927827 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 35.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927827 bytes = 3363392 } seq expand size = 6494789 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5125 value = 3.3055 dict = 1.00 all = 3.4448 } Zip/UnZip{ index = 0.0951 value = 0.3025 dict = 1.00 all = 0.2903 } ---------------------------- total value len = 9.078129 GB avg = 0.167 KB (by entry num) total key len = 1.492723 GB avg = 0.028 KB total ukey len = 1.034951 GB avg = 0.015 KB total ukey num = 0.067404078 Billion total entry num = 0.054211807 Billion write speed all = 0.297582684 MB/sec (with seq num) write speed all = 0.285373644 MB/sec (without seq num) 2024/11/23-18:47:14.665124 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275638: 927827 keys, 38962984 bytes, temperature: kWarm 2024/11/23-18:47:14.665190 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358834665155, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275638, "file_size": 38962984, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3736072251, "largest_seqno": 4193644228, "table_properties": {"data_size": 32744592, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979156, "raw_average_key_size": 28, "raw_value_size": 108238622, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927827, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358831, "oldest_key_time": 0, "file_creation_time": 1732358831, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275638, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:14.701623 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275639.sst) /dev/shm/ToplingTemp/Topling-155195-WmuPUl 2024/11/23-18:47:15.733342 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275639.sst): first pass time = 1.03's, 130.096'MB/sec 2024/11/23-18:47:17.135231 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275639.sst): old prealloc_size = 147639500, real_size = 36590048 2024/11/23-18:47:17.135784 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275639.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932619 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0346 GB avg-val = 115.91 avg-zval = 37.06 2024/11/23-18:47:17.136436 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275639.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:17.910730 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275639.sst): second pass time = 1.40's, 77.159'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 51.500'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3453.136'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.361'MB/sec write SST data time = 0.13's, 308.647'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33112.142'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.159'MB/sec zip pipeline throughput = 77.159'MB/sec entries = 932619 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.91 avg-zval = 37.06 usrkeys = 932619 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.61 avg-zval = 37.06 TagRS{ kinds = RS_Key0_TagN bytes = 145840 } TagArray{ size = 625827 bytes = 2346864 } seq expand size = 4380789 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.3772 value = 3.1281 dict = 1.00 all = 3.3607 } Zip/UnZip{ index = 0.0964 value = 0.3197 dict = 1.00 all = 0.2976 } ---------------------------- total value len = 9.190614 GB avg = 0.167 KB (by entry num) total key len = 1.518836 GB avg = 0.028 KB total ukey len = 1.053603 GB avg = 0.015 KB total ukey num = 0.068336697 Billion total entry num = 0.055144426 Billion write speed all = 0.301456460 MB/sec (with seq num) write speed all = 0.289038535 MB/sec (without seq num) 2024/11/23-18:47:17.960478 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275639: 932619 keys, 39937312 bytes, temperature: kWarm 2024/11/23-18:47:17.960547 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358837960511, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275639, "file_size": 39937312, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34559248, "index_size": 1797440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26113332, "raw_average_key_size": 28, "raw_value_size": 108104518, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932619, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358834, "oldest_key_time": 0, "file_creation_time": 1732358834, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275639, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:17.996622 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275640.sst) /dev/shm/ToplingTemp/Topling-155195-fIK0zj 2024/11/23-18:47:18.970007 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275640.sst): first pass time = 0.97's, 137.893'MB/sec 2024/11/23-18:47:20.301944 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275640.sst): old prealloc_size = 147639500, real_size = 33954256 2024/11/23-18:47:20.302498 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275640.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925946 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0319 GB avg-val = 116.95 avg-zval = 34.50 2024/11/23-18:47:20.303292 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275640.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:21.142803 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275640.sst): second pass time = 1.33's, 81.366'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.528'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3433.833'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 300.588'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30777.085'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.366'MB/sec zip pipeline throughput = 81.366'MB/sec entries = 925946 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.95 avg-zval = 34.50 usrkeys = 925946 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.95 avg-zval = 34.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925946 bytes = 3472320 } seq expand size = 6481622 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4458 value = 3.3894 dict = 1.00 all = 3.5061 } Zip/UnZip{ index = 0.0957 value = 0.2950 dict = 1.00 all = 0.2852 } ---------------------------- total value len = 9.305387 GB avg = 0.166 KB (by entry num) total key len = 1.544763 GB avg = 0.028 KB total ukey len = 1.072122 GB avg = 0.015 KB total ukey num = 0.069262643 Billion total entry num = 0.056070372 Billion write speed all = 0.305389173 MB/sec (with seq num) write speed all = 0.292763884 MB/sec (without seq num) 2024/11/23-18:47:21.189145 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275640: 925946 keys, 38281280 bytes, temperature: kWarm 2024/11/23-18:47:21.189214 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358841189178, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275640, "file_size": 38281280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3276005914, "largest_seqno": 4012775935, "table_properties": {"data_size": 31949696, "index_size": 1772864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926488, "raw_average_key_size": 28, "raw_value_size": 108291292, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925946, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358837, "oldest_key_time": 0, "file_creation_time": 1732358837, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275640, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:21.225272 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275641.sst) /dev/shm/ToplingTemp/Topling-155195-3jwN1h 2024/11/23-18:47:22.212290 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275641.sst): first pass time = 0.99's, 135.988'MB/sec 2024/11/23-18:47:23.536303 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275641.sst): old prealloc_size = 147639500, real_size = 35739104 2024/11/23-18:47:23.536818 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275641.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927442 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 116.72 avg-zval = 36.37 2024/11/23-18:47:23.537612 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275641.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:24.367041 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275641.sst): second pass time = 1.32's, 81.810'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.098'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3612.818'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 316.775'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32539.495'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.810'MB/sec zip pipeline throughput = 81.810'MB/sec entries = 927442 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.72 avg-zval = 36.37 usrkeys = 927442 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.72 avg-zval = 36.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927442 bytes = 3362000 } seq expand size = 6492094 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4442 value = 3.2092 dict = 1.00 all = 3.3592 } Zip/UnZip{ index = 0.0957 value = 0.3116 dict = 1.00 all = 0.2977 } ---------------------------- total value len = 9.420129 GB avg = 0.165 KB (by entry num) total key len = 1.570731 GB avg = 0.028 KB total ukey len = 1.090671 GB avg = 0.016 KB total ukey num = 0.070190085 Billion total entry num = 0.056997814 Billion write speed all = 0.309321533 MB/sec (with seq num) write speed all = 0.296488578 MB/sec (without seq num) 2024/11/23-18:47:24.417439 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275641: 927442 keys, 39955332 bytes, temperature: kWarm 2024/11/23-18:47:24.417509 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358844417474, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275641, "file_size": 39955332, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33731088, "index_size": 1776000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25968376, "raw_average_key_size": 28, "raw_value_size": 108249404, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927442, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358841, "oldest_key_time": 0, "file_creation_time": 1732358841, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275641, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:24.453930 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275642.sst) /dev/shm/ToplingTemp/Topling-155195-EuAdqi 2024/11/23-18:47:25.479973 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275642.sst): first pass time = 1.03's, 130.816'MB/sec 2024/11/23-18:47:26.791537 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275642.sst): old prealloc_size = 147639500, real_size = 32503200 2024/11/23-18:47:26.792104 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275642.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928520 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.55 avg-zval = 32.85 2024/11/23-18:47:26.792913 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275642.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:27.630786 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275642.sst): second pass time = 1.31's, 82.564'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.896'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.697'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 285.957'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28329.975'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.564'MB/sec zip pipeline throughput = 82.564'MB/sec entries = 928520 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 32.85 usrkeys = 928520 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 32.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928520 bytes = 3481968 } seq expand size = 6499640 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5147 value = 3.5476 dict = 1.00 all = 3.6434 } Zip/UnZip{ index = 0.0951 value = 0.2819 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 9.534848 GB avg = 0.165 KB (by entry num) total key len = 1.596730 GB avg = 0.028 KB total ukey len = 1.109241 GB avg = 0.016 KB total ukey num = 0.071118605 Billion total entry num = 0.057926334 Billion write speed all = 0.313253046 MB/sec (with seq num) write speed all = 0.300212235 MB/sec (without seq num) 2024/11/23-18:47:27.675373 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275642: 928520 keys, 36838512 bytes, temperature: kWarm 2024/11/23-18:47:27.675441 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358847675407, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275642, "file_size": 36838512, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30504720, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25998560, "raw_average_key_size": 28, "raw_value_size": 108219308, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928520, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358844, "oldest_key_time": 0, "file_creation_time": 1732358844, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275642, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:27.711643 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275643.sst) /dev/shm/ToplingTemp/Topling-155195-0VjmYk 2024/11/23-18:47:28.731292 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275643.sst): first pass time = 1.02's, 131.636'MB/sec 2024/11/23-18:47:30.078599 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275643.sst): old prealloc_size = 147639500, real_size = 34383040 2024/11/23-18:47:30.079211 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275643.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927855 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.65 avg-zval = 34.89 2024/11/23-18:47:30.079932 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275643.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:30.894490 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275643.sst): second pass time = 1.35's, 80.407'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 46.796'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3004.156'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 300.839'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34734.682'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.407'MB/sec zip pipeline throughput = 80.407'MB/sec entries = 927855 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.65 avg-zval = 34.89 usrkeys = 927855 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.27 avg-zval = 34.89 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 744858 bytes = 2793232 } seq expand size = 5214006 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4451 value = 3.3433 dict = 1.00 all = 3.5155 } Zip/UnZip{ index = 0.0957 value = 0.2991 dict = 1.00 all = 0.2845 } ---------------------------- total value len = 9.648299 GB avg = 0.164 KB (by entry num) total key len = 1.622710 GB avg = 0.028 KB total ukey len = 1.127799 GB avg = 0.016 KB total ukey num = 0.072046460 Billion total entry num = 0.058854189 Billion write speed all = 0.317147660 MB/sec (with seq num) write speed all = 0.303899179 MB/sec (without seq num) 2024/11/23-18:47:30.940273 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275643: 927855 keys, 38178960 bytes, temperature: kWarm 2024/11/23-18:47:30.940341 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358850940306, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275643, "file_size": 38178960, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 32374256, "index_size": 1776640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979940, "raw_average_key_size": 28, "raw_value_size": 108237842, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927855, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358847, "oldest_key_time": 0, "file_creation_time": 1732358847, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275643, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:30.981575 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275644.sst) /dev/shm/ToplingTemp/Topling-155195-ggzPCk 2024/11/23-18:47:31.869499 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275644.sst): first pass time = 0.89's, 151.166'MB/sec 2024/11/23-18:47:33.176580 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275644.sst): old prealloc_size = 147639500, real_size = 31454272 2024/11/23-18:47:33.177134 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275644.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0294 GB avg-val = 115.79 avg-zval = 31.54 2024/11/23-18:47:33.178630 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275644.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:34.017332 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275644.sst): second pass time = 1.31's, 82.772'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.450'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3327.823'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 279.494'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35601.229'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.772'MB/sec zip pipeline throughput = 82.772'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 31.54 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 31.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5138 value = 3.6706 dict = 1.00 all = 3.7608 } Zip/UnZip{ index = 0.0951 value = 0.2724 dict = 1.00 all = 0.2659 } ---------------------------- total value len = 9.762915 GB avg = 0.163 KB (by entry num) total key len = 1.648847 GB avg = 0.028 KB total ukey len = 1.146468 GB avg = 0.016 KB total ukey num = 0.072979917 Billion total entry num = 0.059787646 Billion write speed all = 0.321079977 MB/sec (with seq num) write speed all = 0.307622551 MB/sec (without seq num) 2024/11/23-18:47:34.062965 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275644: 933457 keys, 35688510 bytes, temperature: kWarm 2024/11/23-18:47:34.063049 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358854063014, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275644, "file_size": 35688510, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 29445040, "index_size": 1775680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358850, "oldest_key_time": 0, "file_creation_time": 1732358850, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275644, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:34.100040 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275645.sst) /dev/shm/ToplingTemp/Topling-155195-JhOuNk 2024/11/23-18:47:34.963186 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275645.sst): first pass time = 0.86's, 155.505'MB/sec 2024/11/23-18:47:36.275301 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275645.sst): old prealloc_size = 147639500, real_size = 33336032 2024/11/23-18:47:36.275828 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275645.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931033 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0313 GB avg-val = 116.16 avg-zval = 33.64 2024/11/23-18:47:36.277388 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275645.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:37.124114 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275645.sst): second pass time = 1.31's, 82.496'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.254'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3511.149'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 291.152'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27959.249'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.496'MB/sec zip pipeline throughput = 82.496'MB/sec entries = 931033 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.16 avg-zval = 33.64 usrkeys = 931033 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.16 avg-zval = 33.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931033 bytes = 3375008 } seq expand size = 6517231 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4455 value = 3.4530 dict = 1.00 all = 3.5732 } Zip/UnZip{ index = 0.0957 value = 0.2896 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 9.877581 GB avg = 0.163 KB (by entry num) total key len = 1.674916 GB avg = 0.028 KB total ukey len = 1.165088 GB avg = 0.016 KB total ukey num = 0.073910950 Billion total entry num = 0.060718679 Billion write speed all = 0.325011273 MB/sec (with seq num) write speed all = 0.311345479 MB/sec (without seq num) 2024/11/23-18:47:37.169995 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275645: 931033 keys, 37562732 bytes, temperature: kWarm 2024/11/23-18:47:37.170065 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358857170029, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275645, "file_size": 37562732, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776268, "table_properties": {"data_size": 31320464, "index_size": 1782656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26068924, "raw_average_key_size": 28, "raw_value_size": 108148938, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931033, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358854, "oldest_key_time": 0, "file_creation_time": 1732358854, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275645, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:37.206518 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275646.sst) /dev/shm/ToplingTemp/Topling-155195-4Rweul 2024/11/23-18:47:38.199187 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275646.sst): first pass time = 0.99's, 135.215'MB/sec 2024/11/23-18:47:39.523176 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275646.sst): old prealloc_size = 147639500, real_size = 34558768 2024/11/23-18:47:39.523723 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275646.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924742 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 117.14 avg-zval = 35.21 2024/11/23-18:47:39.525292 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275646.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:40.364156 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275646.sst): second pass time = 1.32's, 81.895'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.235'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3364.818'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 306.568'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36489.686'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.895'MB/sec zip pipeline throughput = 81.895'MB/sec entries = 924742 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.14 avg-zval = 35.21 usrkeys = 924742 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.14 avg-zval = 35.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924742 bytes = 3467792 } seq expand size = 6473194 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4420 value = 3.3273 dict = 1.00 all = 3.4519 } Zip/UnZip{ index = 0.0958 value = 0.3005 dict = 1.00 all = 0.2897 } ---------------------------- total value len = 9.992379 GB avg = 0.162 KB (by entry num) total key len = 1.700808 GB avg = 0.028 KB total ukey len = 1.183583 GB avg = 0.016 KB total ukey num = 0.074835692 Billion total entry num = 0.061643421 Billion write speed all = 0.328939408 MB/sec (with seq num) write speed all = 0.315066749 MB/sec (without seq num) 2024/11/23-18:47:40.415305 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275646: 924742 keys, 38881892 bytes, temperature: kWarm 2024/11/23-18:47:40.415378 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358860415339, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275646, "file_size": 38881892, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012774796, "table_properties": {"data_size": 32556192, "index_size": 1771200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25892776, "raw_average_key_size": 28, "raw_value_size": 108325000, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924742, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358857, "oldest_key_time": 0, "file_creation_time": 1732358857, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275646, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:40.458896 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275647.sst) /dev/shm/ToplingTemp/Topling-155195-MDoFqj 2024/11/23-18:47:41.304173 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275647.sst): first pass time = 0.85's, 158.795'MB/sec 2024/11/23-18:47:42.613361 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275647.sst): old prealloc_size = 147639500, real_size = 32458800 2024/11/23-18:47:42.613951 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275647.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933794 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.73 avg-zval = 32.60 2024/11/23-18:47:42.615486 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275647.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:43.475756 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275647.sst): second pass time = 1.31's, 82.600'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.551'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3161.691'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 286.550'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32865.153'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.600'MB/sec zip pipeline throughput = 82.600'MB/sec entries = 933794 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.73 avg-zval = 32.60 usrkeys = 933794 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.73 avg-zval = 32.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933794 bytes = 3385024 } seq expand size = 6536558 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4453 value = 3.5506 dict = 1.00 all = 3.6577 } Zip/UnZip{ index = 0.0957 value = 0.2816 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 10.106987 GB avg = 0.162 KB (by entry num) total key len = 1.726955 GB avg = 0.028 KB total ukey len = 1.202259 GB avg = 0.016 KB total ukey num = 0.075769486 Billion total entry num = 0.062577215 Billion write speed all = 0.332869814 MB/sec (with seq num) write speed all = 0.318788240 MB/sec (without seq num) 2024/11/23-18:47:43.524857 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275647: 933794 keys, 36694806 bytes, temperature: kWarm 2024/11/23-18:47:43.524928 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358863524893, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275647, "file_size": 36694806, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 30437216, "index_size": 1787968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26146232, "raw_average_key_size": 28, "raw_value_size": 108071628, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933794, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358860, "oldest_key_time": 0, "file_creation_time": 1732358860, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275647, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:43.565089 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275648.sst) /dev/shm/ToplingTemp/Topling-155195-EbBYpl 2024/11/23-18:47:44.586824 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275648.sst): first pass time = 1.02's, 131.368'MB/sec 2024/11/23-18:47:45.885429 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275648.sst): old prealloc_size = 147639500, real_size = 33690368 2024/11/23-18:47:45.885936 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275648.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923043 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.41 avg-zval = 34.33 2024/11/23-18:47:45.887471 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275648.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:46.735727 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275648.sst): second pass time = 1.30's, 83.522'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.703'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3655.725'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 299.423'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34082.908'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.522'MB/sec zip pipeline throughput = 83.522'MB/sec entries = 923043 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.41 avg-zval = 34.33 usrkeys = 923043 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.41 avg-zval = 34.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923043 bytes = 3461424 } seq expand size = 6461301 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.4439 value = 3.4196 dict = 1.00 all = 3.5311 } Zip/UnZip{ index = 0.0957 value = 0.2924 dict = 1.00 all = 0.2832 } ---------------------------- total value len = 10.221821 GB avg = 0.161 KB (by entry num) total key len = 1.752800 GB avg = 0.028 KB total ukey len = 1.220720 GB avg = 0.016 KB total ukey num = 0.076692529 Billion total entry num = 0.063500258 Billion write speed all = 0.336796008 MB/sec (with seq num) write speed all = 0.322508034 MB/sec (without seq num) 2024/11/23-18:47:46.786153 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275648: 923043 keys, 38009678 bytes, temperature: kWarm 2024/11/23-18:47:46.786223 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358866786187, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275648, "file_size": 38009678, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 31691824, "index_size": 1767616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25845204, "raw_average_key_size": 28, "raw_value_size": 108372598, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923043, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358863, "oldest_key_time": 0, "file_creation_time": 1732358863, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275648, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:46.821620 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275649.sst) /dev/shm/ToplingTemp/Topling-155195-NfP3ni 2024/11/23-18:47:47.851910 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275649.sst): first pass time = 1.03's, 130.277'MB/sec 2024/11/23-18:47:49.198155 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275649.sst): old prealloc_size = 147639500, real_size = 33094528 2024/11/23-18:47:49.198714 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275649.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928185 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.60 avg-zval = 33.48 2024/11/23-18:47:49.200275 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275649.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:50.055672 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275649.sst): second pass time = 1.35's, 80.448'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.649'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3334.972'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 296.015'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 24641.993'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.448'MB/sec zip pipeline throughput = 80.448'MB/sec entries = 928185 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.60 avg-zval = 33.48 usrkeys = 928185 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.60 avg-zval = 33.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928185 bytes = 3480704 } seq expand size = 6497295 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3763 value = 3.4830 dict = 1.00 all = 3.5860 } Zip/UnZip{ index = 0.0964 value = 0.2871 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 10.336547 GB avg = 0.160 KB (by entry num) total key len = 1.778789 GB avg = 0.028 KB total ukey len = 1.239284 GB avg = 0.016 KB total ukey num = 0.077620714 Billion total entry num = 0.064428443 Billion write speed all = 0.340721918 MB/sec (with seq num) write speed all = 0.326226450 MB/sec (without seq num) 2024/11/23-18:47:50.102925 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275649: 928185 keys, 37428470 bytes, temperature: kWarm 2024/11/23-18:47:50.103011 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358870102958, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275649, "file_size": 37428470, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 31073264, "index_size": 1789056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989180, "raw_average_key_size": 28, "raw_value_size": 108228550, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928185, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358866, "oldest_key_time": 0, "file_creation_time": 1732358866, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275649, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:50.139148 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275650.sst) /dev/shm/ToplingTemp/Topling-155195-AbP74k 2024/11/23-18:47:51.154936 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275650.sst): first pass time = 1.02's, 132.136'MB/sec 2024/11/23-18:47:52.573125 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275650.sst): old prealloc_size = 147639500, real_size = 35368976 2024/11/23-18:47:52.573693 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275650.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929423 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0334 GB avg-val = 116.41 avg-zval = 35.90 2024/11/23-18:47:52.574076 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275650.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:53.337459 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275650.sst): second pass time = 1.42's, 76.331'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 49.236'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3248.295'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.365'MB/sec write SST data time = 0.13's, 297.679'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33076.310'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.331'MB/sec zip pipeline throughput = 76.331'MB/sec entries = 929423 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.41 avg-zval = 35.90 usrkeys = 929423 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.33 avg-zval = 35.90 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 387601 bytes = 1259712 } seq expand size = 2713207 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5119 value = 3.2424 dict = 1.00 all = 3.5666 } Zip/UnZip{ index = 0.0951 value = 0.3084 dict = 1.00 all = 0.2804 } ---------------------------- total value len = 10.447454 GB avg = 0.160 KB (by entry num) total key len = 1.804813 GB avg = 0.028 KB total ukey len = 1.257872 GB avg = 0.016 KB total ukey num = 0.078550137 Billion total entry num = 0.065357866 Billion write speed all = 0.344541057 MB/sec (with seq num) write speed all = 0.329837839 MB/sec (without seq num) 2024/11/23-18:47:53.385691 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275650: 929423 keys, 37631423 bytes, temperature: kWarm 2024/11/23-18:47:53.385760 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358873385725, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275650, "file_size": 37631423, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 33368128, "index_size": 1768320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023844, "raw_average_key_size": 28, "raw_value_size": 108193982, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929423, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358870, "oldest_key_time": 0, "file_creation_time": 1732358870, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275650, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:53.422111 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275651.sst) /dev/shm/ToplingTemp/Topling-155195-W6OUzj 2024/11/23-18:47:54.391746 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275651.sst): first pass time = 0.97's, 138.427'MB/sec 2024/11/23-18:47:55.870736 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275651.sst): old prealloc_size = 147639500, real_size = 37006320 2024/11/23-18:47:55.871334 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275651.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926689 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0350 GB avg-val = 116.84 avg-zval = 37.76 2024/11/23-18:47:55.871654 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275651.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:56.585954 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275651.sst): second pass time = 1.48's, 73.242'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 54.021'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3098.474'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.392'MB/sec write SST data time = 0.12's, 314.254'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31199.573'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.242'MB/sec zip pipeline throughput = 73.242'MB/sec entries = 926689 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.84 avg-zval = 37.76 usrkeys = 926689 avg-key = 20.00 avg-zkey = 1.93 avg-val = 118.10 avg-zval = 37.76 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 166918 bytes = 625952 } seq expand size = 1168426 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0350 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.3777 value = 3.0945 dict = 1.00 all = 3.4740 } Zip/UnZip{ index = 0.0964 value = 0.3232 dict = 1.00 all = 0.2879 } ---------------------------- total value len = 10.556893 GB avg = 0.159 KB (by entry num) total key len = 1.830760 GB avg = 0.028 KB total ukey len = 1.276406 GB avg = 0.016 KB total ukey num = 0.079476826 Billion total entry num = 0.066284555 Billion write speed all = 0.348316379 MB/sec (with seq num) write speed all = 0.333406051 MB/sec (without seq num) 2024/11/23-18:47:56.633825 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275651: 926689 keys, 38635277 bytes, temperature: kWarm 2024/11/23-18:47:56.633908 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358876633863, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275651, "file_size": 38635277, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4015237319, "table_properties": {"data_size": 34988512, "index_size": 1785920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947292, "raw_average_key_size": 28, "raw_value_size": 108270482, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926689, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358873, "oldest_key_time": 0, "file_creation_time": 1732358873, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275651, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:56.670299 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275652.sst) /dev/shm/ToplingTemp/Topling-155195-BrPUFi 2024/11/23-18:47:57.697818 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275652.sst): first pass time = 1.03's, 130.629'MB/sec 2024/11/23-18:47:59.018769 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275652.sst): old prealloc_size = 147639500, real_size = 34736096 2024/11/23-18:47:59.019334 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275652.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923161 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0327 GB avg-val = 117.39 avg-zval = 35.47 2024/11/23-18:47:59.020095 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275652.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:47:59.856011 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275652.sst): second pass time = 1.32's, 82.088'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.375'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3229.057'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.12's, 310.912'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33823.202'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.088'MB/sec zip pipeline throughput = 82.088'MB/sec entries = 923161 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 35.47 usrkeys = 923161 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 35.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923161 bytes = 3231072 } seq expand size = 6462127 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5123 value = 3.3091 dict = 1.00 all = 3.4570 } Zip/UnZip{ index = 0.0951 value = 0.3022 dict = 1.00 all = 0.2893 } ---------------------------- total value len = 10.671724 GB avg = 0.159 KB (by entry num) total key len = 1.856609 GB avg = 0.028 KB total ukey len = 1.294869 GB avg = 0.016 KB total ukey num = 0.080399987 Billion total entry num = 0.067207716 Billion write speed all = 0.352239635 MB/sec (with seq num) write speed all = 0.337123038 MB/sec (without seq num) 2024/11/23-18:47:59.903725 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275652: 923161 keys, 38824976 bytes, temperature: kWarm 2024/11/23-18:47:59.903793 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358879903757, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275652, "file_size": 38824976, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4015239764, "largest_seqno": 4193644275, "table_properties": {"data_size": 32748752, "index_size": 1756352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25848508, "raw_average_key_size": 28, "raw_value_size": 108369358, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923161, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358876, "oldest_key_time": 0, "file_creation_time": 1732358876, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275652, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:47:59.947715 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275653.sst) /dev/shm/ToplingTemp/Topling-155195-VFNAQi 2024/11/23-18:48:00.868812 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275653.sst): first pass time = 0.92's, 145.722'MB/sec 2024/11/23-18:48:02.216794 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275653.sst): old prealloc_size = 147639500, real_size = 35202576 2024/11/23-18:48:02.217358 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275653.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929382 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.42 avg-zval = 35.72 2024/11/23-18:48:02.218089 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275653.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:48:03.055741 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275653.sst): second pass time = 1.35's, 80.335'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.578'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3272.044'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 304.377'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 26098.897'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.335'MB/sec zip pipeline throughput = 80.335'MB/sec entries = 929382 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.42 avg-zval = 35.72 usrkeys = 929382 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.42 avg-zval = 35.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929382 bytes = 3020512 } seq expand size = 6505674 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5134 value = 3.2587 dict = 1.00 all = 3.4348 } Zip/UnZip{ index = 0.0951 value = 0.3069 dict = 1.00 all = 0.2911 } ---------------------------- total value len = 10.786425 GB avg = 0.158 KB (by entry num) total key len = 1.882631 GB avg = 0.028 KB total ukey len = 1.313457 GB avg = 0.016 KB total ukey num = 0.081329369 Billion total entry num = 0.068137098 Billion write speed all = 0.356164099 MB/sec (with seq num) write speed all = 0.340839840 MB/sec (without seq num) 2024/11/23-18:48:03.104784 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275653: 929382 keys, 39075384 bytes, temperature: kWarm 2024/11/23-18:48:03.104854 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358883104818, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275653, "file_size": 39075384, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4138852857, "largest_seqno": 4193644322, "table_properties": {"data_size": 33202048, "index_size": 1768000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26022696, "raw_average_key_size": 28, "raw_value_size": 108195172, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929382, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358879, "oldest_key_time": 0, "file_creation_time": 1732358879, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275653, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:48:03.144457 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275654.sst) /dev/shm/ToplingTemp/Topling-155195-rtSZVj 2024/11/23-18:48:04.192194 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275654.sst): first pass time = 1.05's, 128.107'MB/sec 2024/11/23-18:48:05.488117 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275654.sst): old prealloc_size = 147639500, real_size = 34185072 2024/11/23-18:48:05.488665 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275654.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926763 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0322 GB avg-val = 116.82 avg-zval = 34.73 2024/11/23-18:48:05.489424 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275654.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:48:06.333156 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275654.sst): second pass time = 1.30's, 83.598'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.168'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3368.106'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 297.636'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35660.823'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.598'MB/sec zip pipeline throughput = 83.598'MB/sec entries = 926763 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 34.73 usrkeys = 926763 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 34.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926763 bytes = 3127840 } seq expand size = 6487341 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5127 value = 3.3634 dict = 1.00 all = 3.5166 } Zip/UnZip{ index = 0.0951 value = 0.2973 dict = 1.00 all = 0.2844 } ---------------------------- total value len = 10.901181 GB avg = 0.158 KB (by entry num) total key len = 1.908581 GB avg = 0.028 KB total ukey len = 1.331992 GB avg = 0.016 KB total ukey num = 0.082256132 Billion total entry num = 0.069063861 Billion write speed all = 0.360086554 MB/sec (with seq num) write speed all = 0.344555294 MB/sec (without seq num) 2024/11/23-18:48:06.380324 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275654: 926763 keys, 38167258 bytes, temperature: kWarm 2024/11/23-18:48:06.380395 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358886380359, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275654, "file_size": 38167258, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4096159344, "largest_seqno": 4193644365, "table_properties": {"data_size": 32190048, "index_size": 1763136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949364, "raw_average_key_size": 28, "raw_value_size": 108268486, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926763, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358883, "oldest_key_time": 0, "file_creation_time": 1732358883, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275654, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:48:06.421440 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275655.sst) /dev/shm/ToplingTemp/Topling-155195-5y7xck 2024/11/23-18:48:06.653379 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275655.sst): first pass time = 0.23's, 242.376'MB/sec 2024/11/23-18:48:07.212704 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275655.sst): old prealloc_size = 147639500, real_size = 15547136 2024/11/23-18:48:07.213002 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275655.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 390768 prefix = 4 raw-key = 0.0063 GB zip-key = 0.0007 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0453 GB zip-val = 0.0147 GB avg-val = 115.84 avg-zval = 37.63 2024/11/23-18:48:07.213357 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275655.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.056 GB 2024/11/23-18:48:07.559645 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00/275655.sst): second pass time = 0.56's, 81.037'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.35's, 49.534'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2705.395'MB/sec (index lex order gen) rebuild zvType time = 0.29's, 0.334'MB/sec write SST data time = 0.05's, 316.480'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.460'MB, throughput = 14350.842'MB/sec zip my value time = 0.56's, unzip length = 0.045'GB zip my value throughput = 81.037'MB/sec zip pipeline throughput = 81.037'MB/sec entries = 390768 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.84 avg-zval = 37.63 usrkeys = 390768 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.84 avg-zval = 37.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 390768 bytes = 1270016 } seq expand size = 2735376 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0078 GB value = 0.0453 GB dict = 0.46 MB all = 0.0562 GB } __ZipSize{ index = 0.0007 GB value = 0.0147 GB dict = 0.46 MB all = 0.0172 GB } UnZip/Zip{ index = 10.4991 value = 3.0782 dict = 1.00 all = 3.2708 } Zip/UnZip{ index = 0.0952 value = 0.3249 dict = 1.00 all = 0.3057 } ---------------------------- total value len = 10.949182 GB avg = 0.158 KB (by entry num) total key len = 1.919522 GB avg = 0.028 KB total ukey len = 1.339807 GB avg = 0.016 KB total ukey num = 0.082646900 Billion total entry num = 0.069454629 Billion write speed all = 0.361730962 MB/sec (with seq num) write speed all = 0.346112363 MB/sec (without seq num) 2024/11/23-18:48:07.585062 139957257127488 [compaction_job.cc:1948] [default] [JOB 33460] Generated table #275655: 390768 keys, 17184263 bytes, temperature: kWarm 2024/11/23-18:48:07.585134 139957257127488 EVENT_LOG_v1 {"time_micros": 1732358887585098, "cf_name": "default", "job": 33460, "event": "table_file_creation", "file_number": 275655, "file_size": 17184263, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4159804009, "largest_seqno": 4193643267, "table_properties": {"data_size": 14704880, "index_size": 744384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10941504, "raw_average_key_size": 28, "raw_value_size": 45265116, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 390768, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358886, "oldest_key_time": 0, "file_creation_time": 1732358886, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275655, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:48:07.600498 139957257127488 [compaction_job.cc:703] job-33460: subcompact[0], size: 0.703707 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33460/att-00 [275637,275638,275639,275640,275641,275642,275643,275644,275645,275646,275647,275648,275649,275650,275651,275652,275653,275654,275655] 2024/11/23-18:48:07.712085 139957257127488 (Original Log Time 2024/11/23-18:47:07.277414) EVENT_LOG_v1 {"time_micros": 1732358827277368, "job": 33460, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [275590], "files_L3": [275610, 275611, 275541, 275542, 275543, 275544, 275545, 275546, 275547, 275548, 275549, 275550, 275551, 275552, 275553, 275554, 275555, 275556, 275499, 275500], "score": 0.997677, "input_data_size": 724133463, "oldest_snapshot_seqno": 4194090192} 2024/11/23-18:48:07.712089 139957257127488 (Original Log Time 2024/11/23-18:48:07.710548) [compaction_job.cc:2022] [default] [JOB 33460] Compacted 1@2 + 20@3 files to L3 => 703707393 bytes 2024/11/23-18:48:07.712090 139957257127488 (Original Log Time 2024/11/23-18:48:07.712015) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 19 0 0 0] max score 0.03, MB/sec: 12.0 rd, 11.7 wr, level 3, files in(1, 20) out(19 +0 blob) MB in(27.2, 663.4 +0.0 blob) out(671.1 +0.0 blob), read-write-amplify(50.1) write-amplify(24.7) OK, records in: 17179117, records dropped: 82652 output_compression: Snappy 2024/11/23-18:48:07.712094 139957257127488 (Original Log Time 2024/11/23-18:48:07.712058) EVENT_LOG_v1 {"time_micros": 1732358887712039, "cf": "default", "job": 33460, "event": "compaction_finished", "compaction_time_micros": 60323091, "compaction_time_cpu_micros": 25422766, "output_level": 3, "num_output_files": 19, "total_output_size": 703707393, "num_input_records": 17179117, "num_output_records": 17096465, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 488450815, "file_range_sync_nanos": 0, "file_fsync_nanos": 13856, "file_prepare_write_nanos": 53073, "lsm_state": [0, 0, 0, 19, 0, 0, 0]} 2024/11/23-18:48:07.829906 139957257127488 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732358827,"executesMs":60570,"compactionJobId":33460,"attempt":0,"compactionInputRawBytes":2483863882,"compactionInputZipBytes":724133463,"compactionOutputRawBytes":0,"compactionOutputZipBytes":703707393,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-21acc96b53cdaf9f1f6773a4e5d4d518-35265aed276d2401-00"}}, response = 2024/11/23-18:48:07.829966 139957257127488 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33460/att-00: olev 3, work 60.570 s, result 5.978 ms, install 1.561 ms, input{raw 2.313 GiB zip 690.587 MiB}