2024/11/22-15:08:20.677810 140024869324352 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00: fork to child time = 0.028470 sec 2024/11/22-15:08:20.678655 140024869324352 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00 2024/11/22-15:08:20.687153 140024869324352 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00 2024/11/22-15:08:20.688580 140024869324352 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-15:08:20.689004 140024869324352 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-15:08:20.750339 140024869324352 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-02199/att-00/MANIFEST-206347 2024/11/22-15:08:20.765112 140024869324352 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-02199/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 210122, last_sequence is 3318054452, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 209940 2024/11/22-15:08:20.765119 140024869324352 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-15:08:20.765315 140024869324352 [version_set.cc:5721] Creating manifest 210122 2024/11/22-15:08:20.797598 140024869324352 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-15:08:20.818399 140024869324352 [compaction_job.cc:2437] [default] [JOB 2199] Compacting 1@2 + 12@3 files to L3, score 0.84, subcompactions 1 : 1 2024/11/22-15:08:20.818419 140024869324352 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 1793 Base level 2, inputs: [210083(27MB)], [210111(39MB) 210112(3205KB) 210027(38MB) 210028(38MB) 210029(35MB) 210030(35MB) 210031(37MB) 210032(38MB) 210051(38MB) 210052(35MB) 210053(37MB) 210115(38MB)] 2024/11/22-15:08:20.824656 140024869324352 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-15:08:20.824696 140024869324352 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-15:08:20.824715 140024869324352 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-15:08:20.824720 140024869324352 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-15:08:20.824730 140024869324352 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-15:08:21.312541 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210123.sst) /dev/shm/ToplingTemp/Topling-165387-hgk7iY 2024/11/22-15:08:22.132638 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210123.sst): first pass time = 0.82's, 163.672'MB/sec 2024/11/22-15:08:24.121086 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210123.sst): old prealloc_size = 147639500, real_size = 37099920 2024/11/22-15:08:24.121743 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210123.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922606 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1084 GB zip-val = 0.0351 GB avg-val = 117.48 avg-zval = 38.07 2024/11/22-15:08:24.122588 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210123.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:08:25.312747 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210123.sst): second pass time = 1.99's, 54.560'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.19's, 34.469'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2753.415'MB/sec (index lex order gen) rebuild zvType time = 0.98's, 0.235'MB/sec write SST data time = 0.21's, 197.141'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36140.665'MB/sec zip my value time = 1.99's, unzip length = 0.108'GB zip my value throughput = 54.560'MB/sec zip pipeline throughput = 54.560'MB/sec entries = 922606 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.48 avg-zval = 38.07 usrkeys = 922606 avg-key = 20.00 avg-zkey = 1.89 avg-val = 124.48 avg-zval = 38.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922606 bytes = 3113808 } seq expand size = 6458242 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0351 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.5823 value = 3.0857 dict = 1.00 all = 3.2676 } Zip/UnZip{ index = 0.0945 value = 0.3241 dict = 1.00 all = 0.3060 } ---------------------------- total value len = 0.114843 GB avg = 0.124 KB (by entry num) total key len = 0.025833 GB avg = 0.028 KB total ukey len = 0.018452 GB avg = 0.020 KB total ukey num = 0.000922606 Billion total entry num = 0.000922606 Billion write speed all = 0.006235025 MB/sec (with seq num) write speed all = 0.005907891 MB/sec (without seq num) 2024/11/22-15:08:25.349197 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210123: 922606 keys, 41075736 bytes, temperature: kWarm 2024/11/22-15:08:25.349283 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259305349231, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210123, "file_size": 41075736, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3317689621, "table_properties": {"data_size": 35125376, "index_size": 1743680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25832968, "raw_average_key_size": 28, "raw_value_size": 108384764, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922606, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259301, "oldest_key_time": 0, "file_creation_time": 1732259301, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210123, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:08:25.387062 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210124.sst) /dev/shm/ToplingTemp/Topling-165387-GMUiz0 2024/11/22-15:08:26.721269 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210124.sst): first pass time = 1.33's, 100.601'MB/sec 2024/11/22-15:08:28.379108 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210124.sst): old prealloc_size = 147639500, real_size = 36461056 2024/11/22-15:08:28.379748 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210124.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923338 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0345 GB avg-val = 117.36 avg-zval = 37.32 2024/11/22-15:08:28.380559 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210124.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:08:29.248983 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210124.sst): second pass time = 1.66's, 65.395'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 46.450'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2868.271'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.15's, 272.707'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31584.638'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.395'MB/sec zip pipeline throughput = 65.395'MB/sec entries = 923338 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.36 avg-zval = 37.32 usrkeys = 923338 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.22 avg-zval = 37.32 TagRS{ kinds = RS_Key0_TagN bytes = 144360 } TagArray{ size = 905026 bytes = 2941344 } seq expand size = 6335182 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.4454 value = 3.1444 dict = 1.00 all = 3.3218 } Zip/UnZip{ index = 0.0957 value = 0.3180 dict = 1.00 all = 0.3010 } ---------------------------- total value len = 0.229543 GB avg = 0.124 KB (by entry num) total key len = 0.051686 GB avg = 0.028 KB total ukey len = 0.036919 GB avg = 0.020 KB total ukey num = 0.001845944 Billion total entry num = 0.001845944 Billion write speed all = 0.012462426 MB/sec (with seq num) write speed all = 0.011808014 MB/sec (without seq num) 2024/11/22-15:08:29.284308 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210124: 923338 keys, 40404940 bytes, temperature: kWarm 2024/11/22-15:08:29.284372 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259309284339, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210124, "file_size": 40404940, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700579, "table_properties": {"data_size": 34462128, "index_size": 1767936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25853464, "raw_average_key_size": 28, "raw_value_size": 108364380, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923338, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259305, "oldest_key_time": 0, "file_creation_time": 1732259305, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210124, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:08:29.321173 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210125.sst) /dev/shm/ToplingTemp/Topling-165387-k7m8jZ 2024/11/22-15:08:30.273734 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210125.sst): first pass time = 0.95's, 140.908'MB/sec 2024/11/22-15:08:32.148000 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210125.sst): old prealloc_size = 147639500, real_size = 36507056 2024/11/22-15:08:32.148574 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210125.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922060 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0345 GB avg-val = 117.56 avg-zval = 37.43 2024/11/22-15:08:32.149293 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210125.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:08:33.094193 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210125.sst): second pass time = 1.87's, 57.858'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 42.561'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3213.593'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.23's, 171.988'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 36154.344'MB/sec zip my value time = 1.87's, unzip length = 0.108'GB zip my value throughput = 57.858'MB/sec zip pipeline throughput = 128.114'MB/sec entries = 922060 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.56 avg-zval = 37.43 usrkeys = 922060 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.01 avg-zval = 37.43 TagRS{ kinds = RS_Key0_TagN bytes = 144160 } TagArray{ size = 848899 bytes = 2758944 } seq expand size = 5942293 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0403 GB } UnZip/Zip{ index = 10.4438 value = 3.1411 dict = 1.00 all = 3.3330 } Zip/UnZip{ index = 0.0958 value = 0.3184 dict = 1.00 all = 0.3000 } ---------------------------- total value len = 0.343885 GB avg = 0.124 KB (by entry num) total key len = 0.077504 GB avg = 0.028 KB total ukey len = 0.055360 GB avg = 0.020 KB total ukey num = 0.002768004 Billion total entry num = 0.002768004 Billion write speed all = 0.018670322 MB/sec (with seq num) write speed all = 0.017689196 MB/sec (without seq num) 2024/11/22-15:08:33.136327 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210125: 922060 keys, 40269106 bytes, temperature: kWarm 2024/11/22-15:08:33.136412 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259313136368, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210125, "file_size": 40269106, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700489, "table_properties": {"data_size": 34510624, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25817680, "raw_average_key_size": 28, "raw_value_size": 108400168, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922060, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259309, "oldest_key_time": 0, "file_creation_time": 1732259309, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210125, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:08:33.204390 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210126.sst) /dev/shm/ToplingTemp/Topling-165387-l1acp1 2024/11/22-15:08:34.158670 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210126.sst): first pass time = 0.95's, 140.657'MB/sec 2024/11/22-15:08:35.927302 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210126.sst): old prealloc_size = 147639500, real_size = 36488880 2024/11/22-15:08:35.927999 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210126.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927281 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.74 avg-zval = 37.20 2024/11/22-15:08:35.928170 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210126.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:08:36.754481 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210126.sst): second pass time = 1.77's, 61.235'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 45.402'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2623.076'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.342'MB/sec write SST data time = 0.15's, 252.794'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31229.653'MB/sec zip my value time = 1.77's, unzip length = 0.108'GB zip my value throughput = 61.235'MB/sec zip pipeline throughput = 61.235'MB/sec entries = 927281 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 37.20 usrkeys = 927281 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.92 avg-zval = 37.20 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 23123 bytes = 63600 } seq expand size = 161861 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5151 value = 3.1384 dict = 1.00 all = 3.5739 } Zip/UnZip{ index = 0.0951 value = 0.3186 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 0.452301 GB avg = 0.122 KB (by entry num) total key len = 0.103468 GB avg = 0.028 KB total ukey len = 0.073906 GB avg = 0.020 KB total ukey num = 0.003695285 Billion total entry num = 0.003695285 Billion write speed all = 0.024620237 MB/sec (with seq num) write speed all = 0.023310645 MB/sec (without seq num) 2024/11/22-15:08:36.788788 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210126: 927281 keys, 37555355 bytes, temperature: kWarm 2024/11/22-15:08:36.788856 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259316788821, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210126, "file_size": 37555355, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 34493152, "index_size": 1763712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963868, "raw_average_key_size": 28, "raw_value_size": 108253942, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927281, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259313, "oldest_key_time": 0, "file_creation_time": 1732259313, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210126, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:08:36.823434 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210127.sst) /dev/shm/ToplingTemp/Topling-165387-lZmzf1 2024/11/22-15:08:37.789829 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210127.sst): first pass time = 0.97's, 138.890'MB/sec 2024/11/22-15:08:39.389498 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210127.sst): old prealloc_size = 147639500, real_size = 36471152 2024/11/22-15:08:39.390166 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210127.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927076 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.19 2024/11/22-15:08:39.390223 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210127.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:08:40.370982 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210127.sst): second pass time = 1.60's, 67.725'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 38.035'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2756.154'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.24's, 155.713'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33244.501'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.725'MB/sec zip pipeline throughput = 67.725'MB/sec entries = 927076 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 37.19 usrkeys = 927076 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 37.19 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5124 value = 3.1402 dict = 1.00 all = 3.5955 } Zip/UnZip{ index = 0.0951 value = 0.3185 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 0.560560 GB avg = 0.121 KB (by entry num) total key len = 0.129426 GB avg = 0.028 KB total ukey len = 0.092447 GB avg = 0.020 KB total ukey num = 0.004622361 Billion total entry num = 0.004622361 Billion write speed all = 0.030561109 MB/sec (with seq num) write speed all = 0.028923228 MB/sec (without seq num) 2024/11/22-15:08:40.407207 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210127: 927076 keys, 37329444 bytes, temperature: kWarm 2024/11/22-15:08:40.407291 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259320407246, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210127, "file_size": 37329444, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34475424, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958128, "raw_average_key_size": 28, "raw_value_size": 108259612, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927076, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259316, "oldest_key_time": 0, "file_creation_time": 1732259316, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210127, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:08:40.476943 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210128.sst) /dev/shm/ToplingTemp/Topling-165387-rxmPcY 2024/11/22-15:08:42.049829 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210128.sst): first pass time = 1.57's, 85.335'MB/sec 2024/11/22-15:08:43.739158 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210128.sst): old prealloc_size = 147639500, real_size = 36777680 2024/11/22-15:08:43.739699 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210128.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924038 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 117.25 avg-zval = 37.65 2024/11/22-15:08:43.740213 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210128.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:08:44.768157 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210128.sst): second pass time = 1.67's, 64.867'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 38.543'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3368.040'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.262'MB/sec write SST data time = 0.15's, 270.751'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 37105.045'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 64.867'MB/sec zip pipeline throughput = 64.867'MB/sec entries = 924038 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.25 avg-zval = 37.65 usrkeys = 924038 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.63 avg-zval = 37.65 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 577501 bytes = 1876896 } seq expand size = 4042507 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5115 value = 3.1144 dict = 1.00 all = 3.3842 } Zip/UnZip{ index = 0.0951 value = 0.3211 dict = 1.00 all = 0.2955 } ---------------------------- total value len = 0.672948 GB avg = 0.121 KB (by entry num) total key len = 0.155299 GB avg = 0.028 KB total ukey len = 0.110928 GB avg = 0.020 KB total ukey num = 0.005546399 Billion total entry num = 0.005546399 Billion write speed all = 0.036677837 MB/sec (with seq num) write speed all = 0.034712916 MB/sec (without seq num) 2024/11/22-15:08:44.810318 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210128: 924038 keys, 39659561 bytes, temperature: kWarm 2024/11/22-15:08:44.810397 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259324810355, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210128, "file_size": 39659561, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3317689354, "table_properties": {"data_size": 34788352, "index_size": 1758144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25873064, "raw_average_key_size": 28, "raw_value_size": 108344756, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924038, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259320, "oldest_key_time": 0, "file_creation_time": 1732259320, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210128, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:08:44.871508 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210129.sst) /dev/shm/ToplingTemp/Topling-165387-iuDHI1 2024/11/22-15:08:46.471382 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210129.sst): first pass time = 1.60's, 83.895'MB/sec 2024/11/22-15:08:48.656837 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210129.sst): old prealloc_size = 147639500, real_size = 36140848 2024/11/22-15:08:48.657356 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210129.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923147 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0342 GB avg-val = 117.39 avg-zval = 37.00 2024/11/22-15:08:48.658075 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210129.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:08:49.770692 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210129.sst): second pass time = 2.18's, 49.611'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 35.910'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3526.019'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.238'MB/sec write SST data time = 0.14's, 277.613'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 26609.021'MB/sec zip my value time = 2.18's, unzip length = 0.108'GB zip my value throughput = 49.611'MB/sec zip pipeline throughput = 72.945'MB/sec entries = 923147 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 37.00 usrkeys = 923147 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 37.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923147 bytes = 3000240 } seq expand size = 6462029 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.5094 value = 3.1731 dict = 1.00 all = 3.3556 } Zip/UnZip{ index = 0.0952 value = 0.3152 dict = 1.00 all = 0.2980 } ---------------------------- total value len = 0.787779 GB avg = 0.122 KB (by entry num) total key len = 0.181147 GB avg = 0.028 KB total ukey len = 0.129391 GB avg = 0.020 KB total ukey num = 0.006469546 Billion total entry num = 0.006469546 Billion write speed all = 0.042898155 MB/sec (with seq num) write speed all = 0.040606699 MB/sec (without seq num) 2024/11/22-15:08:49.807535 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210129: 923147 keys, 39998700 bytes, temperature: kWarm 2024/11/22-15:08:49.807594 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259329807564, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210129, "file_size": 39998700, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3317689116, "table_properties": {"data_size": 34153056, "index_size": 1756800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25848116, "raw_average_key_size": 28, "raw_value_size": 108369638, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923147, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259324, "oldest_key_time": 0, "file_creation_time": 1732259324, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210129, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:08:49.845124 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210130.sst) /dev/shm/ToplingTemp/Topling-165387-cg0J6W 2024/11/22-15:08:50.815557 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210130.sst): first pass time = 0.97's, 138.313'MB/sec 2024/11/22-15:08:52.155851 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210130.sst): old prealloc_size = 147639500, real_size = 36158848 2024/11/22-15:08:52.156380 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210130.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922285 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0342 GB avg-val = 117.53 avg-zval = 37.04 2024/11/22-15:08:52.157110 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210130.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:08:53.017271 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210130.sst): second pass time = 1.34's, 80.943'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 46.454'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3484.625'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.315'MB/sec write SST data time = 0.13's, 310.531'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 30377.260'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.943'MB/sec zip pipeline throughput = 80.943'MB/sec entries = 922285 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.53 avg-zval = 37.04 usrkeys = 922285 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.53 avg-zval = 37.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922285 bytes = 2997440 } seq expand size = 6455995 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4452 value = 3.1729 dict = 1.00 all = 3.3542 } Zip/UnZip{ index = 0.0957 value = 0.3152 dict = 1.00 all = 0.2981 } ---------------------------- total value len = 0.902629 GB avg = 0.122 KB (by entry num) total key len = 0.206971 GB avg = 0.028 KB total ukey len = 0.147837 GB avg = 0.020 KB total ukey num = 0.007391831 Billion total entry num = 0.007391831 Billion write speed all = 0.049119269 MB/sec (with seq num) write speed all = 0.046501525 MB/sec (without seq num) 2024/11/22-15:08:53.051332 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210130: 922285 keys, 40015184 bytes, temperature: kWarm 2024/11/22-15:08:53.051396 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259333051364, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210130, "file_size": 40015184, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701445, "largest_seqno": 3317689513, "table_properties": {"data_size": 34162160, "index_size": 1765952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25823980, "raw_average_key_size": 28, "raw_value_size": 108393786, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922285, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259329, "oldest_key_time": 0, "file_creation_time": 1732259329, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210130, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:08:53.090608 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210131.sst) /dev/shm/ToplingTemp/Topling-165387-YJ6ESY 2024/11/22-15:08:54.062579 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210131.sst): first pass time = 0.97's, 138.095'MB/sec 2024/11/22-15:08:55.472682 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210131.sst): old prealloc_size = 147639500, real_size = 35226624 2024/11/22-15:08:55.473301 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210131.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081919 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0329 GB avg-val = 96.06 avg-zval = 30.39 2024/11/22-15:08:55.473624 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210131.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:08:56.115752 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210131.sst): second pass time = 1.41's, 73.735'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 56.583'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3465.410'MB/sec (index lex order gen) rebuild zvType time = 0.50's, 0.539'MB/sec write SST data time = 0.14's, 258.261'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.035'MB, throughput = 31605.568'MB/sec zip my value time = 1.41's, unzip length = 0.104'GB zip my value throughput = 73.735'MB/sec zip pipeline throughput = 73.735'MB/sec entries = 1081919 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.06 avg-zval = 30.39 usrkeys = 1081919 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.54 avg-zval = 30.39 TagRS{ kinds = RS_Key0_TagN bytes = 169160 } TagArray{ size = 74626 bytes = 214560 } seq expand size = 522382 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0329 GB dict = 1.04 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4474 value = 3.1602 dict = 1.00 all = 3.6887 } Zip/UnZip{ index = 0.0957 value = 0.3164 dict = 1.00 all = 0.2711 } ---------------------------- total value len = 1.007076 GB avg = 0.119 KB (by entry num) total key len = 0.237265 GB avg = 0.028 KB total ukey len = 0.169475 GB avg = 0.020 KB total ukey num = 0.008473750 Billion total entry num = 0.008473750 Billion write speed all = 0.055076332 MB/sec (with seq num) write speed all = 0.052075848 MB/sec (without seq num) 2024/11/22-15:08:56.152472 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210131: 1081919 keys, 36386135 bytes, temperature: kWarm 2024/11/22-15:08:56.152536 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259336152504, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210131, "file_size": 36386135, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3317688773, "table_properties": {"data_size": 32884784, "index_size": 2071168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30293732, "raw_average_key_size": 28, "raw_value_size": 103924123, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081919, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259333, "oldest_key_time": 0, "file_creation_time": 1732259333, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210131, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:08:56.189027 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210132.sst) /dev/shm/ToplingTemp/Topling-165387-WEUa7Z 2024/11/22-15:08:57.133900 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210132.sst): first pass time = 0.94's, 142.055'MB/sec 2024/11/22-15:08:58.512073 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210132.sst): old prealloc_size = 147639500, real_size = 36506816 2024/11/22-15:08:58.512615 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210132.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924276 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 117.21 avg-zval = 37.35 2024/11/22-15:08:58.513118 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210132.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:08:59.286916 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210132.sst): second pass time = 1.38's, 78.645'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 50.691'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3381.501'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.358'MB/sec write SST data time = 0.13's, 303.155'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33030.016'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.645'MB/sec zip pipeline throughput = 78.645'MB/sec entries = 924276 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.21 avg-zval = 37.35 usrkeys = 924276 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.33 avg-zval = 37.35 TagRS{ kinds = RS_Key0_TagN bytes = 144520 } TagArray{ size = 543166 bytes = 1765312 } seq expand size = 3802162 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5142 value = 3.1386 dict = 1.00 all = 3.4172 } Zip/UnZip{ index = 0.0951 value = 0.3186 dict = 1.00 all = 0.2926 } ---------------------------- total value len = 1.119216 GB avg = 0.119 KB (by entry num) total key len = 0.263145 GB avg = 0.028 KB total ukey len = 0.187961 GB avg = 0.020 KB total ukey num = 0.009398026 Billion total entry num = 0.009398026 Billion write speed all = 0.061176712 MB/sec (with seq num) write speed all = 0.057849416 MB/sec (without seq num) 2024/11/22-15:08:59.324309 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210132: 924276 keys, 39276723 bytes, temperature: kWarm 2024/11/22-15:08:59.324382 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259339324340, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210132, "file_size": 39276723, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3317687989, "table_properties": {"data_size": 34517424, "index_size": 1758144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25879728, "raw_average_key_size": 28, "raw_value_size": 108338112, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924276, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259336, "oldest_key_time": 0, "file_creation_time": 1732259336, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210132, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:08:59.365036 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210133.sst) /dev/shm/ToplingTemp/Topling-165387-co1f7W 2024/11/22-15:09:00.264378 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210133.sst): first pass time = 0.90's, 149.247'MB/sec 2024/11/22-15:09:01.618593 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210133.sst): old prealloc_size = 147639500, real_size = 36050368 2024/11/22-15:09:01.619149 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210133.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922122 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0341 GB avg-val = 117.55 avg-zval = 36.94 2024/11/22-15:09:01.621508 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210133.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:09:02.484658 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210133.sst): second pass time = 1.35's, 80.114'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 46.078'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.361'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.314'MB/sec write SST data time = 0.13's, 303.001'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31339.350'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.114'MB/sec zip pipeline throughput = 80.114'MB/sec entries = 922122 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.55 avg-zval = 36.94 usrkeys = 922122 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.55 avg-zval = 36.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922122 bytes = 2996912 } seq expand size = 6454854 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.5108 value = 3.1821 dict = 1.00 all = 3.3633 } Zip/UnZip{ index = 0.0951 value = 0.3143 dict = 1.00 all = 0.2973 } ---------------------------- total value len = 1.234069 GB avg = 0.120 KB (by entry num) total key len = 0.288964 GB avg = 0.028 KB total ukey len = 0.206403 GB avg = 0.020 KB total ukey num = 0.010320148 Billion total entry num = 0.010320148 Billion write speed all = 0.067392681 MB/sec (with seq num) write speed all = 0.063739432 MB/sec (without seq num) 2024/11/22-15:09:02.519970 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210133: 922122 keys, 39906008 bytes, temperature: kWarm 2024/11/22-15:09:02.520030 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259342519998, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210133, "file_size": 39906008, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3317689240, "table_properties": {"data_size": 34065008, "index_size": 1754624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25819416, "raw_average_key_size": 28, "raw_value_size": 108398444, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922122, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259339, "oldest_key_time": 0, "file_creation_time": 1732259339, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210133, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:09:02.557824 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210134.sst) /dev/shm/ToplingTemp/Topling-165387-OtdorX 2024/11/22-15:09:02.583927 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210134.sst): first pass time = 0.03's, 199.928'MB/sec 2024/11/22-15:09:02.634873 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210134.sst): old prealloc_size = 147639500, real_size = 1687920 2024/11/22-15:09:02.634944 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210134.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 35682 prefix = 4 raw-key = 0.0006 GB zip-key = 0.0001 GB avg-key = 16.00 avg-zkey = 1.96 raw-val = 0.0042 GB zip-val = 0.0016 GB avg-val = 118.00 avg-zval = 45.09 2024/11/22-15:09:02.635004 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210134.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0001 GB, waited 0.000 sec, Key+Value = 0.005 GB 2024/11/22-15:09:02.665030 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00/210134.sst): second pass time = 0.05's, 83.478'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.03's, 60.509'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1393.248'MB/sec (index lex order gen) rebuild zvType time = 0.02's, 0.358'MB/sec write SST data time = 0.01's, 351.271'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.041'MB, throughput = 1264.441'MB/sec zip my value time = 0.05's, unzip length = 0.004'GB zip my value throughput = 83.478'MB/sec zip pipeline throughput = 83.478'MB/sec entries = 35682 avg-key = 28.00 avg-zkey = 1.96 avg-val = 118.00 avg-zval = 45.09 usrkeys = 35682 avg-key = 20.00 avg-zkey = 1.96 avg-val = 125.00 avg-zval = 45.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 35682 bytes = 102608 } seq expand size = 249774 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0007 GB value = 0.0042 GB dict = 0.04 MB all = 0.0052 GB } __ZipSize{ index = 0.0001 GB value = 0.0016 GB dict = 0.04 MB all = 0.0018 GB } UnZip/Zip{ index = 10.2206 value = 2.6168 dict = 1.00 all = 2.8561 } Zip/UnZip{ index = 0.0978 value = 0.3821 dict = 1.00 all = 0.3501 } ---------------------------- total value len = 1.238529 GB avg = 0.120 KB (by entry num) total key len = 0.289963 GB avg = 0.028 KB total ukey len = 0.207117 GB avg = 0.020 KB total ukey num = 0.010355830 Billion total entry num = 0.010355830 Billion write speed all = 0.067633711 MB/sec (with seq num) write speed all = 0.063967861 MB/sec (without seq num) 2024/11/22-15:09:02.672260 140024869324352 [compaction_job.cc:1948] [default] [JOB 2199] Generated table #210134: 35682 keys, 1824001 bytes, temperature: kWarm 2024/11/22-15:09:02.672330 140024869324352 EVENT_LOG_v1 {"time_micros": 1732259342672285, "cf_name": "default", "job": 2199, "event": "table_file_creation", "file_number": 210134, "file_size": 1824001, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3300423305, "largest_seqno": 3307219373, "table_properties": {"data_size": 1608992, "index_size": 69824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 999096, "raw_average_key_size": 28, "raw_value_size": 4210476, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 35682, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732259342, "oldest_key_time": 0, "file_creation_time": 1732259342, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 210134, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:09:02.681414 140024869324352 [compaction_job.cc:703] job-02199: subcompact[0], size: 0.433701 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-02199/att-00 [210123,210124,210125,210126,210127,210128,210129,210130,210131,210132,210133,210134] 2024/11/22-15:09:02.758883 140024869324352 (Original Log Time 2024/11/22-15:08:20.818471) EVENT_LOG_v1 {"time_micros": 1732259300818433, "job": 2199, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [210083], "files_L3": [210111, 210112, 210027, 210028, 210029, 210030, 210031, 210032, 210051, 210052, 210053, 210115], "score": 0.843148, "input_data_size": 465027413, "oldest_snapshot_seqno": 3318054400} 2024/11/22-15:09:02.758887 140024869324352 (Original Log Time 2024/11/22-15:09:02.758185) [compaction_job.cc:2022] [default] [JOB 2199] Compacted 1@2 + 12@3 files to L3 => 433700893 bytes 2024/11/22-15:09:02.758888 140024869324352 (Original Log Time 2024/11/22-15:09:02.758821) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 12 0 0 0] max score 0.02, MB/sec: 11.1 rd, 10.4 wr, level 3, files in(1, 12) out(12 +0 blob) MB in(27.1, 416.4 +0.0 blob) out(413.6 +0.0 blob), read-write-amplify(31.6) write-amplify(15.3) OK, records in: 10464357, records dropped: 108527 output_compression: Snappy 2024/11/22-15:09:02.758896 140024869324352 (Original Log Time 2024/11/22-15:09:02.758865) EVENT_LOG_v1 {"time_micros": 1732259342758845, "cf": "default", "job": 2199, "event": "compaction_finished", "compaction_time_micros": 41862949, "compaction_time_cpu_micros": 17848716, "output_level": 3, "num_output_files": 12, "total_output_size": 433700893, "num_input_records": 10464357, "num_output_records": 10355830, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 314742797, "file_range_sync_nanos": 0, "file_fsync_nanos": 11844, "file_prepare_write_nanos": 26896, "lsm_state": [0, 0, 0, 12, 0, 0, 0]} 2024/11/22-15:09:02.880770 140024869324352 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732259300,"executesMs":42075,"compactionJobId":2199,"attempt":0,"compactionInputRawBytes":1519414486,"compactionInputZipBytes":465027413,"compactionOutputRawBytes":0,"compactionOutputZipBytes":433700893,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c60b0e4f06648dbe9f5e3601ec61284c-410d11f48b2bd621-00"}}, response = 2024/11/22-15:09:02.880853 140024869324352 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-02199/att-00: olev 3, work 42.075 s, result 4.912 ms, install 0.732 ms, input{raw 1.415 GiB zip 443.485 MiB}