2024/11/20-23:53:38.780827 140008417130048 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00: fork to child time = 0.008947 sec 2024/11/20-23:53:38.781352 140008417130048 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00 2024/11/20-23:53:38.787801 140008417130048 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00 2024/11/20-23:53:38.789507 140008417130048 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-23:53:38.789736 140008417130048 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-23:53:38.835763 140008417130048 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-21939/att-00/MANIFEST-049091 2024/11/20-23:53:38.891742 140008417130048 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-21939/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 87475, last_sequence is 1336361125, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 87276 2024/11/20-23:53:38.891747 140008417130048 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-23:53:38.891831 140008417130048 [version_set.cc:5724] Creating manifest 87475 2024/11/20-23:53:38.923399 140008417130048 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-23:53:38.943145 140008417130048 [compaction_job.cc:2437] [default] [JOB 21939] Compacting 1@2 + 14@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/20-23:53:38.943156 140008417130048 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 16386 Base level 2, inputs: [87423(27MB)], [87337(36MB) 87338(36MB) 87339(36MB) 87367(37MB) 87368(36MB) 87369(37MB) 87370(40MB) 87371(36MB) 87372(37MB) 87373(38MB) 87374(37MB) 87375(36MB) 87376(36MB) 87463(36MB)] 2024/11/20-23:53:38.951029 140008417130048 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-23:53:38.951059 140008417130048 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-23:53:38.951075 140008417130048 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-23:53:38.951081 140008417130048 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-23:53:38.951089 140008417130048 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-23:53:39.484656 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087476.sst) /dev/shm/ToplingTemp/Topling-1169467-MF93vM 2024/11/20-23:53:40.076762 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087476.sst): first pass time = 0.59's, 226.694'MB/sec 2024/11/20-23:53:41.389598 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087476.sst): old prealloc_size = 147639500, real_size = 34489520 2024/11/20-23:53:41.390297 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087476.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 842595 prefix = 4 raw-key = 0.0135 GB zip-key = 0.0016 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0327 GB avg-val = 128.40 avg-zval = 38.78 2024/11/20-23:53:41.392006 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087476.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0021 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:53:42.257686 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087476.sst): second pass time = 1.31's, 82.470'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 44.669'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2420.475'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.293'MB/sec write SST data time = 0.15's, 259.783'MB/sec dict compress time = 2842445865.57's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 22585.306'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.470'MB/sec zip pipeline throughput = 155.584'MB/sec entries = 929691 avg-key = 28.00 avg-zkey = 1.72 avg-val = 116.37 avg-zval = 35.15 usrkeys = 842595 avg-key = 20.00 avg-zkey = 1.90 avg-val = 137.15 avg-zval = 38.78 TagRS{ kinds = RS_KeyN_Tag1 bytes = 158520 } TagArray{ size = 929691 bytes = 3253936 } seq expand size = 6682029 multi value expand size = 696768 cnt WriteAppend = 13 UnZipSize{ index = 0.0169 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0016 GB value = 0.0327 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5127 value = 3.3109 dict = 1.00 all = 3.4614 } Zip/UnZip{ index = 0.0951 value = 0.3020 dict = 1.00 all = 0.2889 } ---------------------------- total value len = 563.181298 GB avg = 0.331 KB (by entry num) total key len = 44.340438 GB avg = 0.026 KB total ukey len = 32.810994 GB avg = 0.014 KB total ukey num = 2.321857171 Billion total entry num = 1.701446289 Billion write speed all = 28.929731640 MB/sec (with seq num) write speed all = 28.281558820 MB/sec (without seq num) 2024/11/20-23:53:42.335575 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87476: 929691 keys, 38776004 bytes, temperature: kWarm 2024/11/20-23:53:42.335699 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118022335640, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87476, "file_size": 38776004, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1069697601, "largest_seqno": 1335962701, "table_properties": {"data_size": 32675680, "index_size": 1603008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26031348, "raw_average_key_size": 28, "raw_value_size": 108186494, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929691, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118019, "oldest_key_time": 0, "file_creation_time": 1732118019, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87476, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:53:42.384515 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087477.sst) /dev/shm/ToplingTemp/Topling-1169467-Mjs3PN 2024/11/20-23:53:43.287207 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087477.sst): first pass time = 0.90's, 148.694'MB/sec 2024/11/20-23:53:44.470476 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087477.sst): old prealloc_size = 147639500, real_size = 33667552 2024/11/20-23:53:44.470948 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087477.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 841388 prefix = 4 raw-key = 0.0135 GB zip-key = 0.0016 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0319 GB avg-val = 128.60 avg-zval = 37.86 2024/11/20-23:53:44.471775 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087477.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0021 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:53:45.328745 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087477.sst): second pass time = 1.18's, 91.511'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 44.220'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3626.291'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.292'MB/sec write SST data time = 0.14's, 278.265'MB/sec dict compress time = 0.00's, 1079420.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36869.215'MB/sec zip my value time = 1.18's, unzip length = 0.108'GB zip my value throughput = 91.511'MB/sec zip pipeline throughput = 91.511'MB/sec entries = 929066 avg-key = 28.00 avg-zkey = 1.72 avg-val = 116.47 avg-zval = 34.29 usrkeys = 841388 avg-key = 20.00 avg-zkey = 1.90 avg-val = 137.37 avg-zval = 37.86 TagRS{ kinds = RS_KeyN_Tag1 bytes = 158400 } TagArray{ size = 929066 bytes = 3251744 } seq expand size = 6678818 multi value expand size = 701424 cnt WriteAppend = 12 UnZipSize{ index = 0.0168 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0016 GB value = 0.0319 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5123 value = 3.3966 dict = 1.00 all = 3.5365 } Zip/UnZip{ index = 0.0951 value = 0.2944 dict = 1.00 all = 0.2828 } ---------------------------- total value len = 563.296882 GB avg = 0.331 KB (by entry num) total key len = 44.366452 GB avg = 0.026 KB total ukey len = 32.827821 GB avg = 0.014 KB total ukey num = 2.322698559 Billion total entry num = 1.702375355 Billion write speed all = 28.932243313 MB/sec (with seq num) write speed all = 28.283811390 MB/sec (without seq num) 2024/11/20-23:53:45.377350 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87477: 929066 keys, 37952140 bytes, temperature: kWarm 2024/11/20-23:53:45.377407 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118025377378, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87477, "file_size": 37952140, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1069697639, "largest_seqno": 1335962429, "table_properties": {"data_size": 31856272, "index_size": 1600768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26013848, "raw_average_key_size": 28, "raw_value_size": 108203880, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929066, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118022, "oldest_key_time": 0, "file_creation_time": 1732118022, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87477, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:53:45.410431 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087478.sst) /dev/shm/ToplingTemp/Topling-1169467-exVs4N 2024/11/20-23:53:46.249663 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087478.sst): first pass time = 0.84's, 159.935'MB/sec 2024/11/20-23:53:47.414006 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087478.sst): old prealloc_size = 147639500, real_size = 33612256 2024/11/20-23:53:47.414518 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087478.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 851489 prefix = 4 raw-key = 0.0136 GB zip-key = 0.0016 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0318 GB avg-val = 127.12 avg-zval = 37.32 2024/11/20-23:53:47.415362 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087478.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0021 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:53:48.330978 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087478.sst): second pass time = 1.16's, 93.024'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.323'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3234.248'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.278'MB/sec write SST data time = 0.15's, 249.611'MB/sec dict compress time = 0.00's, 1080092.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31464.794'MB/sec zip my value time = 1.16's, unzip length = 0.108'GB zip my value throughput = 93.024'MB/sec zip pipeline throughput = 93.024'MB/sec entries = 927857 avg-key = 28.00 avg-zkey = 1.75 avg-val = 116.65 avg-zval = 34.25 usrkeys = 851489 avg-key = 20.00 avg-zkey = 1.90 avg-val = 135.64 avg-zval = 37.32 TagRS{ kinds = RS_KeyN_Tag1 bytes = 158360 } TagArray{ size = 927857 bytes = 3247520 } seq expand size = 6647735 multi value expand size = 610944 cnt WriteAppend = 13 UnZipSize{ index = 0.0170 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0016 GB value = 0.0318 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5137 value = 3.4059 dict = 1.00 all = 3.5422 } Zip/UnZip{ index = 0.0951 value = 0.2936 dict = 1.00 all = 0.2823 } ---------------------------- total value len = 563.412378 GB avg = 0.331 KB (by entry num) total key len = 44.392432 GB avg = 0.026 KB total ukey len = 32.844851 GB avg = 0.014 KB total ukey num = 2.323550048 Billion total entry num = 1.703303212 Billion write speed all = 28.934843289 MB/sec (with seq num) write speed all = 28.286150673 MB/sec (without seq num) 2024/11/20-23:53:48.375722 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87478: 927857 keys, 37890796 bytes, temperature: kWarm 2024/11/20-23:53:48.375787 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118028375756, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87478, "file_size": 37890796, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1156808633, "largest_seqno": 1335961786, "table_properties": {"data_size": 31779408, "index_size": 1619776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979996, "raw_average_key_size": 28, "raw_value_size": 108237754, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927857, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118025, "oldest_key_time": 0, "file_creation_time": 1732118025, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87478, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:53:48.412298 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087479.sst) /dev/shm/ToplingTemp/Topling-1169467-XewmmQ 2024/11/20-23:53:49.399397 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087479.sst): first pass time = 0.99's, 135.976'MB/sec 2024/11/20-23:53:50.572737 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087479.sst): old prealloc_size = 147639500, real_size = 34918192 2024/11/20-23:53:50.573265 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087479.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 860449 prefix = 4 raw-key = 0.0138 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1081 GB zip-val = 0.0330 GB avg-val = 125.65 avg-zval = 38.37 2024/11/20-23:53:50.574099 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087479.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0022 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:53:51.441879 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087479.sst): second pass time = 1.17's, 92.203'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 45.083'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3331.635'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.296'MB/sec write SST data time = 0.14's, 276.765'MB/sec dict compress time = 0.00's, 1077866.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35610.744'MB/sec zip my value time = 1.17's, unzip length = 0.108'GB zip my value throughput = 92.203'MB/sec zip pipeline throughput = 92.203'MB/sec entries = 932204 avg-key = 28.00 avg-zkey = 1.81 avg-val = 115.98 avg-zval = 35.41 usrkeys = 860449 avg-key = 20.00 avg-zkey = 1.97 avg-val = 133.41 avg-zval = 38.37 TagRS{ kinds = RS_KeyN_TagN bytes = 304912 } TagArray{ size = 851712 bytes = 3087472 } seq expand size = 6105494 multi value expand size = 574040 cnt WriteAppend = 85 UnZipSize{ index = 0.0172 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0330 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.1748 value = 3.2751 dict = 1.00 all = 3.4255 } Zip/UnZip{ index = 0.0983 value = 0.3053 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 563.527174 GB avg = 0.331 KB (by entry num) total key len = 44.418533 GB avg = 0.026 KB total ukey len = 32.862060 GB avg = 0.014 KB total ukey num = 2.324410497 Billion total entry num = 1.704235416 Billion write speed all = 28.937265285 MB/sec (with seq num) write speed all = 28.288313752 MB/sec (without seq num) 2024/11/20-23:53:51.490246 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87479: 932204 keys, 39182266 bytes, temperature: kWarm 2024/11/20-23:53:51.490305 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118031490276, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87479, "file_size": 39182266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1335962815, "table_properties": {"data_size": 33011552, "index_size": 1691328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26101712, "raw_average_key_size": 28, "raw_value_size": 108116148, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932204, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118028, "oldest_key_time": 0, "file_creation_time": 1732118028, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87479, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:53:51.524915 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087480.sst) /dev/shm/ToplingTemp/Topling-1169467-XG7dDO 2024/11/20-23:53:52.403483 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087480.sst): first pass time = 0.88's, 152.774'MB/sec 2024/11/20-23:53:53.738662 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087480.sst): old prealloc_size = 147639500, real_size = 36481744 2024/11/20-23:53:53.739394 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087480.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926791 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.82 avg-zval = 37.15 2024/11/20-23:53:53.739893 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087480.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:53:54.469870 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087480.sst): second pass time = 1.33's, 81.262'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 53.368'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2573.515'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.398'MB/sec write SST data time = 0.15's, 262.809'MB/sec dict compress time = 0.00's, 1080302.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31241.563'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.262'MB/sec zip pipeline throughput = 81.262'MB/sec entries = 926791 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.82 avg-zval = 37.15 usrkeys = 926791 avg-key = 20.00 avg-zkey = 1.97 avg-val = 120.26 avg-zval = 37.15 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 455658 bytes = 1537856 } seq expand size = 3189606 multi value expand size = 0 cnt WriteAppend = 60 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.1750 value = 3.1447 dict = 1.00 all = 3.4395 } Zip/UnZip{ index = 0.0983 value = 0.3180 dict = 1.00 all = 0.2907 } ---------------------------- total value len = 563.638631 GB avg = 0.331 KB (by entry num) total key len = 44.444483 GB avg = 0.026 KB total ukey len = 32.880596 GB avg = 0.014 KB total ukey num = 2.325337288 Billion total entry num = 1.705162207 Billion write speed all = 28.939634650 MB/sec (with seq num) write speed all = 28.290423776 MB/sec (without seq num) 2024/11/20-23:53:54.517805 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87480: 926791 keys, 39022653 bytes, temperature: kWarm 2024/11/20-23:53:54.517854 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118034517825, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87480, "file_size": 39022653, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 978567417, "table_properties": {"data_size": 34428160, "index_size": 1821696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950148, "raw_average_key_size": 28, "raw_value_size": 108267690, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926791, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118031, "oldest_key_time": 0, "file_creation_time": 1732118031, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87480, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:53:54.553585 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087481.sst) /dev/shm/ToplingTemp/Topling-1169467-6vvZTM 2024/11/20-23:53:55.399434 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087481.sst): first pass time = 0.85's, 158.684'MB/sec 2024/11/20-23:53:57.119769 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087481.sst): old prealloc_size = 147639500, real_size = 36403232 2024/11/20-23:53:57.120539 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087481.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926704 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.83 avg-zval = 37.07 2024/11/20-23:53:57.120870 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087481.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:53:58.118321 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087481.sst): second pass time = 1.72's, 62.967'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 38.245'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2437.034'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.18's, 206.488'MB/sec dict compress time = 0.00's, 1080298.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31337.511'MB/sec zip my value time = 1.72's, unzip length = 0.108'GB zip my value throughput = 62.967'MB/sec zip pipeline throughput = 160.783'MB/sec entries = 926704 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.83 avg-zval = 37.07 usrkeys = 926704 avg-key = 20.00 avg-zkey = 1.97 avg-val = 118.59 avg-zval = 37.07 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 231949 bytes = 782848 } seq expand size = 1623643 multi value expand size = 0 cnt WriteAppend = 36 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.1773 value = 3.1519 dict = 1.00 all = 3.5145 } Zip/UnZip{ index = 0.0983 value = 0.3173 dict = 1.00 all = 0.2845 } ---------------------------- total value len = 563.748525 GB avg = 0.330 KB (by entry num) total key len = 44.470431 GB avg = 0.026 KB total ukey len = 32.899130 GB avg = 0.014 KB total ukey num = 2.326263992 Billion total entry num = 1.706088911 Billion write speed all = 28.941074365 MB/sec (with seq num) write speed all = 28.291623432 MB/sec (without seq num) 2024/11/20-23:53:58.170334 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87481: 926704 keys, 38189177 bytes, temperature: kWarm 2024/11/20-23:53:58.170402 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118038170369, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87481, "file_size": 38189177, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 963091028, "table_properties": {"data_size": 34350224, "index_size": 1821120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947712, "raw_average_key_size": 28, "raw_value_size": 108270028, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926704, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118034, "oldest_key_time": 0, "file_creation_time": 1732118034, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87481, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:53:58.215687 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087482.sst) /dev/shm/ToplingTemp/Topling-1169467-Bd06pM 2024/11/20-23:53:59.136141 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087482.sst): first pass time = 0.92's, 145.823'MB/sec 2024/11/20-23:54:00.433952 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087482.sst): old prealloc_size = 147639500, real_size = 38511088 2024/11/20-23:54:00.434728 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087482.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926829 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0365 GB avg-val = 116.81 avg-zval = 39.41 2024/11/20-23:54:00.436136 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087482.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:54:02.064829 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087482.sst): second pass time = 1.30's, 83.480'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.63's, 26.223'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2340.327'MB/sec (index lex order gen) rebuild zvType time = 1.33's, 0.175'MB/sec write SST data time = 0.30's, 141.006'MB/sec dict compress time = 0.00's, 1080214.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30600.096'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.480'MB/sec zip pipeline throughput = 83.480'MB/sec entries = 926829 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.81 avg-zval = 39.41 usrkeys = 926829 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.59 avg-zval = 39.41 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 897126 bytes = 3252096 } seq expand size = 6279882 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0365 GB dict = 1.08 MB all = 0.0428 GB } UnZip/Zip{ index = 10.5818 value = 2.9640 dict = 1.00 all = 3.1384 } Zip/UnZip{ index = 0.0945 value = 0.3374 dict = 1.00 all = 0.3186 } ---------------------------- total value len = 563.863071 GB avg = 0.330 KB (by entry num) total key len = 44.496382 GB avg = 0.026 KB total ukey len = 32.917667 GB avg = 0.014 KB total ukey num = 2.327190821 Billion total entry num = 1.707015740 Billion write speed all = 28.942324733 MB/sec (with seq num) write speed all = 28.292642989 MB/sec (without seq num) 2024/11/20-23:54:02.174641 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87482: 926829 keys, 42766182 bytes, temperature: kWarm 2024/11/20-23:54:02.174733 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118042174688, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87482, "file_size": 42766182, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1331715475, "table_properties": {"data_size": 36527456, "index_size": 1751744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951212, "raw_average_key_size": 28, "raw_value_size": 108266634, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118038, "oldest_key_time": 0, "file_creation_time": 1732118038, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87482, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:54:02.269681 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087483.sst) /dev/shm/ToplingTemp/Topling-1169467-0X3PHN 2024/11/20-23:54:03.318890 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087483.sst): first pass time = 1.05's, 127.931'MB/sec 2024/11/20-23:54:04.521607 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087483.sst): old prealloc_size = 147639500, real_size = 34464512 2024/11/20-23:54:04.522237 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087483.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 860565 prefix = 4 raw-key = 0.0138 GB zip-key = 0.0016 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 125.83 avg-zval = 37.90 2024/11/20-23:54:04.523058 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087483.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0022 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:54:05.597586 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087483.sst): second pass time = 1.20's, 90.101'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 36.000'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2723.522'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.242'MB/sec write SST data time = 0.18's, 209.861'MB/sec dict compress time = 0.00's, 1080428.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33897.907'MB/sec zip my value time = 1.20's, unzip length = 0.108'GB zip my value throughput = 90.101'MB/sec zip pipeline throughput = 95.383'MB/sec entries = 926031 avg-key = 28.00 avg-zkey = 1.77 avg-val = 116.94 avg-zval = 35.22 usrkeys = 860565 avg-key = 20.00 avg-zkey = 1.90 avg-val = 134.13 avg-zval = 37.90 TagRS{ kinds = RS_KeyN_Tag1 bytes = 158240 } TagArray{ size = 926031 bytes = 3241120 } seq expand size = 6613149 multi value expand size = 523728 cnt WriteAppend = 13 UnZipSize{ index = 0.0172 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0016 GB value = 0.0326 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5132 value = 3.3205 dict = 1.00 all = 3.4651 } Zip/UnZip{ index = 0.0951 value = 0.3012 dict = 1.00 all = 0.2886 } ---------------------------- total value len = 563.978497 GB avg = 0.330 KB (by entry num) total key len = 44.522311 GB avg = 0.026 KB total ukey len = 32.934878 GB avg = 0.014 KB total ukey num = 2.328051386 Billion total entry num = 1.707941771 Billion write speed all = 28.944184985 MB/sec (with seq num) write speed all = 28.294260030 MB/sec (without seq num) 2024/11/20-23:54:05.648929 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87483: 926031 keys, 38734716 bytes, temperature: kWarm 2024/11/20-23:54:05.649027 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118045648976, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87483, "file_size": 38734716, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187855341, "largest_seqno": 1331717282, "table_properties": {"data_size": 32612080, "index_size": 1637120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928868, "raw_average_key_size": 28, "raw_value_size": 108288986, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926031, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118042, "oldest_key_time": 0, "file_creation_time": 1732118042, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87483, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:54:05.720820 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087484.sst) /dev/shm/ToplingTemp/Topling-1169467-JF3gTN 2024/11/20-23:54:07.457078 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087484.sst): first pass time = 1.74's, 77.306'MB/sec 2024/11/20-23:54:08.730013 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087484.sst): old prealloc_size = 147639500, real_size = 35611584 2024/11/20-23:54:08.730617 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087484.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 864346 prefix = 4 raw-key = 0.0138 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 125.17 avg-zval = 39.04 2024/11/20-23:54:08.731430 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087484.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0022 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:54:09.769479 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087484.sst): second pass time = 1.27's, 85.037'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 38.379'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2929.090'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.242'MB/sec write SST data time = 0.15's, 274.614'MB/sec dict compress time = 0.00's, 1079384.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35520.074'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.037'MB/sec zip pipeline throughput = 85.037'MB/sec entries = 929679 avg-key = 28.00 avg-zkey = 1.78 avg-val = 116.37 avg-zval = 36.29 usrkeys = 864346 avg-key = 20.00 avg-zkey = 1.91 avg-val = 133.45 avg-zval = 39.04 TagRS{ kinds = RS_KeyN_Tag1 bytes = 158840 } TagArray{ size = 929679 bytes = 3253888 } seq expand size = 6638419 multi value expand size = 522664 cnt WriteAppend = 13 UnZipSize{ index = 0.0173 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0337 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.4462 value = 3.2064 dict = 1.00 all = 3.3644 } Zip/UnZip{ index = 0.0957 value = 0.3119 dict = 1.00 all = 0.2972 } ---------------------------- total value len = 564.093845 GB avg = 0.330 KB (by entry num) total key len = 44.548342 GB avg = 0.026 KB total ukey len = 32.952165 GB avg = 0.014 KB total ukey num = 2.328915732 Billion total entry num = 1.708871450 Billion write speed all = 28.945165905 MB/sec (with seq num) write speed all = 28.295016195 MB/sec (without seq num) 2024/11/20-23:54:09.819780 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87484: 929679 keys, 39893160 bytes, temperature: kWarm 2024/11/20-23:54:09.819853 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118049819820, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87484, "file_size": 39893160, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187855426, "largest_seqno": 1331716698, "table_properties": {"data_size": 33740464, "index_size": 1654848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26031012, "raw_average_key_size": 28, "raw_value_size": 108186822, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929679, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118045, "oldest_key_time": 0, "file_creation_time": 1732118045, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87484, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:54:09.860583 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087485.sst) /dev/shm/ToplingTemp/Topling-1169467-DcrNDM 2024/11/20-23:54:10.802025 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087485.sst): first pass time = 0.94's, 142.572'MB/sec 2024/11/20-23:54:12.043267 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087485.sst): old prealloc_size = 147639500, real_size = 35537824 2024/11/20-23:54:12.043829 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087485.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 843923 prefix = 4 raw-key = 0.0135 GB zip-key = 0.0016 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 128.16 avg-zval = 39.96 2024/11/20-23:54:12.044669 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087485.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0021 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:54:12.889848 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087485.sst): second pass time = 1.24's, 87.201'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 47.049'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3080.366'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.302'MB/sec write SST data time = 0.15's, 270.165'MB/sec dict compress time = 0.00's, 1078006.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37053.793'MB/sec zip my value time = 1.24's, unzip length = 0.108'GB zip my value throughput = 87.201'MB/sec zip pipeline throughput = 87.201'MB/sec entries = 930870 avg-key = 28.00 avg-zkey = 1.73 avg-val = 116.19 avg-zval = 36.22 usrkeys = 843923 avg-key = 20.00 avg-zkey = 1.90 avg-val = 136.91 avg-zval = 39.96 TagRS{ kinds = RS_KeyN_Tag1 bytes = 158720 } TagArray{ size = 930870 bytes = 3258064 } seq expand size = 6689984 multi value expand size = 695576 cnt WriteAppend = 14 UnZipSize{ index = 0.0169 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0016 GB value = 0.0337 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.5099 value = 3.2073 dict = 1.00 all = 3.3697 } Zip/UnZip{ index = 0.0951 value = 0.3118 dict = 1.00 all = 0.2968 } ---------------------------- total value len = 564.209384 GB avg = 0.330 KB (by entry num) total key len = 44.574407 GB avg = 0.026 KB total ukey len = 32.969043 GB avg = 0.014 KB total ukey num = 2.329759655 Billion total entry num = 1.709802320 Billion write speed all = 28.947604437 MB/sec (with seq num) write speed all = 28.297197091 MB/sec (without seq num) 2024/11/20-23:54:12.938427 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87485: 930870 keys, 39830406 bytes, temperature: kWarm 2024/11/20-23:54:12.938496 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118052938463, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87485, "file_size": 39830406, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187851486, "largest_seqno": 1331716806, "table_properties": {"data_size": 33720720, "index_size": 1605952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26064360, "raw_average_key_size": 28, "raw_value_size": 108153408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930870, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118049, "oldest_key_time": 0, "file_creation_time": 1732118049, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87485, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:54:12.977434 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087486.sst) /dev/shm/ToplingTemp/Topling-1169467-IGcOdM 2024/11/20-23:54:13.946288 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087486.sst): first pass time = 0.97's, 138.539'MB/sec 2024/11/20-23:54:15.136737 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087486.sst): old prealloc_size = 147639500, real_size = 35905696 2024/11/20-23:54:15.137336 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087486.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 868423 prefix = 4 raw-key = 0.0139 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0340 GB avg-val = 124.52 avg-zval = 39.17 2024/11/20-23:54:15.138159 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087486.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0022 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:54:15.976018 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087486.sst): second pass time = 1.19's, 90.891'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.889'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2953.318'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.313'MB/sec write SST data time = 0.15's, 277.122'MB/sec dict compress time = 0.00's, 1078244.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35191.880'MB/sec zip my value time = 1.19's, unzip length = 0.108'GB zip my value throughput = 90.891'MB/sec zip pipeline throughput = 90.891'MB/sec entries = 931358 avg-key = 28.00 avg-zkey = 1.80 avg-val = 116.11 avg-zval = 36.52 usrkeys = 868423 avg-key = 20.00 avg-zkey = 1.93 avg-val = 132.76 avg-zval = 39.17 TagRS{ kinds = RS_KeyN_Tag1 bytes = 159200 } TagArray{ size = 931358 bytes = 3259760 } seq expand size = 6645376 multi value expand size = 503480 cnt WriteAppend = 16 UnZipSize{ index = 0.0174 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0340 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.3771 value = 3.1792 dict = 1.00 all = 3.3395 } Zip/UnZip{ index = 0.0964 value = 0.3145 dict = 1.00 all = 0.2994 } ---------------------------- total value len = 564.324673 GB avg = 0.330 KB (by entry num) total key len = 44.600485 GB avg = 0.026 KB total ukey len = 32.986412 GB avg = 0.014 KB total ukey num = 2.330628078 Billion total entry num = 1.710733678 Billion write speed all = 28.950078059 MB/sec (with seq num) write speed all = 28.299411908 MB/sec (without seq num) 2024/11/20-23:54:16.024887 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87486: 931358 keys, 40191348 bytes, temperature: kWarm 2024/11/20-23:54:16.024953 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118056024921, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87486, "file_size": 40191348, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1187851565, "largest_seqno": 1331717093, "table_properties": {"data_size": 34014672, "index_size": 1673728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26078024, "raw_average_key_size": 28, "raw_value_size": 108139712, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931358, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118052, "oldest_key_time": 0, "file_creation_time": 1732118052, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87486, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:54:16.060771 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087487.sst) /dev/shm/ToplingTemp/Topling-1169467-2W6r3L 2024/11/20-23:54:17.033265 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087487.sst): first pass time = 0.97's, 138.019'MB/sec 2024/11/20-23:54:18.218364 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087487.sst): old prealloc_size = 147639500, real_size = 34468896 2024/11/20-23:54:18.218892 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087487.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 851644 prefix = 4 raw-key = 0.0136 GB zip-key = 0.0016 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 127.14 avg-zval = 38.32 2024/11/20-23:54:18.219707 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087487.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0021 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:54:19.079419 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087487.sst): second pass time = 1.18's, 91.424'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 45.000'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3213.709'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.297'MB/sec write SST data time = 0.14's, 270.311'MB/sec dict compress time = 0.00's, 1080116.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31375.919'MB/sec zip my value time = 1.18's, unzip length = 0.108'GB zip my value throughput = 91.424'MB/sec zip pipeline throughput = 91.424'MB/sec entries = 926559 avg-key = 28.00 avg-zkey = 1.75 avg-val = 116.86 avg-zval = 35.22 usrkeys = 851644 avg-key = 20.00 avg-zkey = 1.90 avg-val = 135.63 avg-zval = 38.32 TagRS{ kinds = RS_KeyN_Tag1 bytes = 158160 } TagArray{ size = 926559 bytes = 3242976 } seq expand size = 6635743 multi value expand size = 599320 cnt WriteAppend = 14 UnZipSize{ index = 0.0170 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0016 GB value = 0.0326 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5123 value = 3.3177 dict = 1.00 all = 3.4640 } Zip/UnZip{ index = 0.0951 value = 0.3014 dict = 1.00 all = 0.2887 } ---------------------------- total value len = 564.440182 GB avg = 0.330 KB (by entry num) total key len = 44.626428 GB avg = 0.026 KB total ukey len = 33.003445 GB avg = 0.014 KB total ukey num = 2.331479722 Billion total entry num = 1.711660237 Billion write speed all = 28.952531352 MB/sec (with seq num) write speed all = 28.301608830 MB/sec (without seq num) 2024/11/20-23:54:19.125809 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87487: 926559 keys, 38746180 bytes, temperature: kWarm 2024/11/20-23:54:19.125875 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118059125844, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87487, "file_size": 38746180, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1105231892, "largest_seqno": 1335961852, "table_properties": {"data_size": 32635536, "index_size": 1620288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943652, "raw_average_key_size": 28, "raw_value_size": 108274090, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926559, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118056, "oldest_key_time": 0, "file_creation_time": 1732118056, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87487, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:54:19.161164 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087488.sst) /dev/shm/ToplingTemp/Topling-1169467-Qq4ppP 2024/11/20-23:54:20.013825 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087488.sst): first pass time = 0.85's, 157.419'MB/sec 2024/11/20-23:54:21.150455 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087488.sst): old prealloc_size = 147639500, real_size = 33842400 2024/11/20-23:54:21.151025 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087488.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 846116 prefix = 4 raw-key = 0.0135 GB zip-key = 0.0016 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0320 GB avg-val = 127.80 avg-zval = 37.84 2024/11/20-23:54:21.151886 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087488.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0021 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:54:22.009668 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087488.sst): second pass time = 1.14's, 95.206'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 44.620'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2951.948'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.296'MB/sec write SST data time = 0.14's, 266.340'MB/sec dict compress time = 0.00's, 1078110.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 26975.679'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.206'MB/sec zip pipeline throughput = 95.206'MB/sec entries = 931496 avg-key = 28.00 avg-zkey = 1.73 avg-val = 116.09 avg-zval = 34.38 usrkeys = 846116 avg-key = 20.00 avg-zkey = 1.90 avg-val = 136.39 avg-zval = 37.84 TagRS{ kinds = RS_KeyN_TagN bytes = 304448 } TagArray{ size = 915465 bytes = 3318576 } seq expand size = 6579015 multi value expand size = 683040 cnt WriteAppend = 12 UnZipSize{ index = 0.0169 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0016 GB value = 0.0320 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5134 value = 3.3770 dict = 1.00 all = 3.5010 } Zip/UnZip{ index = 0.0951 value = 0.2961 dict = 1.00 all = 0.2856 } ---------------------------- total value len = 564.555580 GB avg = 0.330 KB (by entry num) total key len = 44.652510 GB avg = 0.026 KB total ukey len = 33.020367 GB avg = 0.014 KB total ukey num = 2.332325838 Billion total entry num = 1.712591733 Billion write speed all = 28.955223485 MB/sec (with seq num) write speed all = 28.304037433 MB/sec (without seq num) 2024/11/20-23:54:22.057270 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87488: 931496 keys, 38337390 bytes, temperature: kWarm 2024/11/20-23:54:22.057330 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118062057299, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87488, "file_size": 38337390, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1335962377, "table_properties": {"data_size": 32021072, "index_size": 1609600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26081888, "raw_average_key_size": 28, "raw_value_size": 108135908, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931496, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118059, "oldest_key_time": 0, "file_creation_time": 1732118059, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87488, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:54:22.092490 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087489.sst) /dev/shm/ToplingTemp/Topling-1169467-nWsVuP 2024/11/20-23:54:22.957978 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087489.sst): first pass time = 0.87's, 155.083'MB/sec 2024/11/20-23:54:24.083209 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087489.sst): old prealloc_size = 147639500, real_size = 34152192 2024/11/20-23:54:24.083740 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087489.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 842747 prefix = 4 raw-key = 0.0135 GB zip-key = 0.0016 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 128.41 avg-zval = 38.36 2024/11/20-23:54:24.084563 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087489.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0021 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-23:54:24.938693 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087489.sst): second pass time = 1.12's, 96.239'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 45.064'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3200.971'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.294'MB/sec write SST data time = 0.14's, 279.323'MB/sec dict compress time = 0.00's, 1079636.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 20458.888'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 96.239'MB/sec zip pipeline throughput = 96.239'MB/sec entries = 928612 avg-key = 28.00 avg-zkey = 1.74 avg-val = 116.54 avg-zval = 34.81 usrkeys = 842747 avg-key = 20.00 avg-zkey = 1.91 avg-val = 137.14 avg-zval = 38.36 TagRS{ kinds = RS_KeyN_Tag1 bytes = 158360 } TagArray{ size = 928612 bytes = 3366240 } seq expand size = 6672014 multi value expand size = 686920 cnt WriteAppend = 15 UnZipSize{ index = 0.0169 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0016 GB value = 0.0323 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4441 value = 3.3475 dict = 1.00 all = 3.4816 } Zip/UnZip{ index = 0.0957 value = 0.2987 dict = 1.00 all = 0.2872 } ---------------------------- total value len = 564.671155 GB avg = 0.330 KB (by entry num) total key len = 44.678511 GB avg = 0.026 KB total ukey len = 33.037222 GB avg = 0.014 KB total ukey num = 2.333168585 Billion total entry num = 1.713520345 Billion write speed all = 28.957921174 MB/sec (with seq num) write speed all = 28.306472723 MB/sec (without seq num) 2024/11/20-23:54:24.984355 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87489: 928612 keys, 38551140 bytes, temperature: kWarm 2024/11/20-23:54:24.984419 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118064984389, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87489, "file_size": 38551140, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1053131113, "largest_seqno": 1335962468, "table_properties": {"data_size": 32327472, "index_size": 1613824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26001136, "raw_average_key_size": 28, "raw_value_size": 108216704, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928612, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118062, "oldest_key_time": 0, "file_creation_time": 1732118062, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87489, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:54:25.018182 140008417130048 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087490.sst) /dev/shm/ToplingTemp/Topling-1169467-06En2M 2024/11/20-23:54:25.149236 140008417130048 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087490.sst): first pass time = 0.13's, 256.665'MB/sec 2024/11/20-23:54:25.426888 140008417130048 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087490.sst): old prealloc_size = 147639500, real_size = 8969424 2024/11/20-23:54:25.427094 140008417130048 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087490.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 191982 prefix = 4 raw-key = 0.0031 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0270 GB zip-val = 0.0086 GB avg-val = 140.89 avg-zval = 44.56 2024/11/20-23:54:25.427356 140008417130048 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087490.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.034 GB 2024/11/20-23:54:25.639496 140008417130048 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00/087490.sst): second pass time = 0.28's, 97.694'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 47.474'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2061.532'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.272'MB/sec write SST data time = 0.04's, 278.642'MB/sec dict compress time = 0.00's, 277864.000'MB/sec z-dict build time = 0.00's, sample length = 0.278'MB, throughput = 4431.219'MB/sec zip my value time = 0.28's, unzip length = 0.027'GB zip my value throughput = 97.694'MB/sec zip pipeline throughput = 97.694'MB/sec entries = 234958 avg-key = 28.00 avg-zkey = 1.56 avg-val = 115.12 avg-zval = 36.41 usrkeys = 191982 avg-key = 20.00 avg-zkey = 1.91 avg-val = 151.69 avg-zval = 44.56 TagRS{ kinds = RS_KeyN_Tag1 bytes = 39800 } TagArray{ size = 234958 bytes = 851744 } seq expand size = 1730658 multi value expand size = 343808 cnt WriteAppend = 14 UnZipSize{ index = 0.0038 GB value = 0.0270 GB dict = 0.28 MB all = 0.0336 GB } __ZipSize{ index = 0.0004 GB value = 0.0086 GB dict = 0.28 MB all = 0.0101 GB } UnZip/Zip{ index = 10.4757 value = 3.1617 dict = 1.00 all = 3.3317 } Zip/UnZip{ index = 0.0955 value = 0.3163 dict = 1.00 all = 0.3002 } ---------------------------- total value len = 564.700277 GB avg = 0.330 KB (by entry num) total key len = 44.685090 GB avg = 0.026 KB total ukey len = 33.041062 GB avg = 0.014 KB total ukey num = 2.333360567 Billion total entry num = 1.713755303 Billion write speed all = 28.958653327 MB/sec (with seq num) write speed all = 28.307137247 MB/sec (without seq num) 2024/11/20-23:54:25.656741 140008417130048 [compaction_job.cc:1948] [default] [JOB 21939] Generated table #87490: 234958 keys, 10092983 bytes, temperature: kWarm 2024/11/20-23:54:25.656793 140008417130048 EVENT_LOG_v1 {"time_micros": 1732118065656765, "cf_name": "default", "job": 21939, "event": "table_file_creation", "file_number": 87490, "file_size": 10092983, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1053131138, "largest_seqno": 1335961936, "table_properties": {"data_size": 8554688, "index_size": 366528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6578824, "raw_average_key_size": 28, "raw_value_size": 27047488, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 234958, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_KeyN_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732118065, "oldest_key_time": 0, "file_creation_time": 1732118065, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 87490, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-23:54:25.668335 140008417130048 [compaction_job.cc:703] job-21939: subcompact[0], size: 0.558157 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21939/att-00 [087476,087477,087478,087479,087480,087481,087482,087483,087484,087485,087486,087487,087488,087489,087490] 2024/11/20-23:54:25.774495 140008417130048 (Original Log Time 2024/11/20-23:53:38.943184) EVENT_LOG_v1 {"time_micros": 1732118018943162, "job": 21939, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [87423], "files_L3": [87337, 87338, 87339, 87367, 87368, 87369, 87370, 87371, 87372, 87373, 87374, 87375, 87376, 87463], "score": 0.995553, "input_data_size": 574607302, "oldest_snapshot_seqno": 1323177126} 2024/11/20-23:54:25.774498 140008417130048 (Original Log Time 2024/11/20-23:54:25.773219) [compaction_job.cc:2022] [default] [JOB 21939] Compacted 1@2 + 14@3 files to L3 => 558156541 bytes 2024/11/20-23:54:25.774499 140008417130048 (Original Log Time 2024/11/20-23:54:25.774425) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 12.3 rd, 11.9 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.2, 520.8 +0.0 blob) out(532.3 +0.0 blob), read-write-amplify(39.7) write-amplify(19.6) OK, records in: 13238705, records dropped: 0 output_compression: Snappy 2024/11/20-23:54:25.774503 140008417130048 (Original Log Time 2024/11/20-23:54:25.774474) EVENT_LOG_v1 {"time_micros": 1732118065774448, "cf": "default", "job": 21939, "event": "compaction_finished", "compaction_time_micros": 46725156, "compaction_time_cpu_micros": 21156075, "output_level": 3, "num_output_files": 15, "total_output_size": 558156541, "num_input_records": 13238705, "num_output_records": 13238705, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 399942509, "file_range_sync_nanos": 0, "file_fsync_nanos": 13411, "file_prepare_write_nanos": 46300, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/20-23:54:25.902835 140008417130048 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732118018,"executesMs":46984,"compactionJobId":21939,"attempt":0,"compactionInputRawBytes":1912675486,"compactionInputZipBytes":574607302,"compactionOutputRawBytes":0,"compactionOutputZipBytes":558156541,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-a21f579e134dcb490b6581fa08850504-bca1c3784907e7e7-00"}}, response = 2024/11/20-23:54:25.902887 140008417130048 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-21939/att-00: olev 3, work 46.985 s, result 7.597 ms, install 1.301 ms, input{raw 1.781 GiB zip 547.988 MiB}