2024/11/20-18:42:55.285523 140008543020608 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00: fork to child time = 0.008555 sec 2024/11/20-18:42:55.286041 140008543020608 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00 2024/11/20-18:42:55.292394 140008543020608 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00 2024/11/20-18:42:55.298807 140008543020608 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-18:42:55.299059 140008543020608 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-18:42:55.343981 140008543020608 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-06591/att-00/MANIFEST-049091 2024/11/20-18:42:55.359013 140008543020608 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-06591/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 58825, last_sequence is 985180042, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 58663 2024/11/20-18:42:55.359019 140008543020608 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-18:42:55.359096 140008543020608 [version_set.cc:5724] Creating manifest 58825 2024/11/20-18:42:55.390063 140008543020608 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-18:42:55.412083 140008543020608 [compaction_job.cc:2437] [default] [JOB 6591] Compacting 1@2 + 11@3 files to L3, score 0.78, subcompactions 1 : 1 2024/11/20-18:42:55.412095 140008543020608 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 6026 Base level 2, inputs: [58820(21MB)], [58735(124MB) 58736(124MB) 58737(124MB) 58738(100MB) 58739(100MB) 58740(100MB) 58741(100MB) 58742(124MB) 58743(100MB) 58744(100MB) 58745(73MB)] 2024/11/20-18:42:55.418034 140008543020608 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-18:42:55.418064 140008543020608 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-18:42:55.418083 140008543020608 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-18:42:55.418090 140008543020608 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-18:42:55.418100 140008543020608 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-18:42:56.742059 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058826.sst) /dev/shm/ToplingTemp/Topling-847461-6vYZrd 2024/11/20-18:42:56.982736 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058826.sst): first pass time = 0.24's, 557.798'MB/sec 2024/11/20-18:42:58.237525 140008543020608 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058826.sst): part-0: reject ZipStore ratio = 0.801 2024/11/20-18:42:58.624526 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058826.sst): old prealloc_size = 147639500, real_size = 130016048 2024/11/20-18:42:58.624743 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058826.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200577 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.74 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.16 avg-zval = 646.22 2024/11/20-18:42:58.625008 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058826.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:42:59.073451 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058826.sst): second pass time = 1.25's, 103.367'MB/sec, value only(96.4% of KV) wait indexing time = 0.39's, remap KeyValue time = 0.45's, 291.023'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1779.773'MB/sec (index lex order gen) rebuild zvType time = 0.38's, 0.134'MB/sec write SST data time = 0.07's, 1778.183'MB/sec dict compress time = 3472328296.23's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.25's, unzip length = 0.129'GB zip my value throughput = 103.367'MB/sec zip pipeline throughput = 103.367'MB/sec entries = 200577 avg-key = 24.00 avg-zkey = 1.74 avg-val = 645.16 avg-zval = 646.22 usrkeys = 200577 avg-key = 16.00 avg-zkey = 1.74 avg-val = 651.87 avg-zval = 646.22 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 192318 bytes = 649088 } seq expand size = 1346226 multi value expand size = 0 cnt WriteAppend = 15807 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1306 GB } UnZip/Zip{ index = 9.2008 value = 0.9984 dict = 0.00 all = 1.0273 } Zip/UnZip{ index = 0.1087 value = 1.0016 dict = 0.00 all = 0.9734 } ---------------------------- total value len = 0.130750 GB avg = 0.652 KB (by entry num) total key len = 0.004814 GB avg = 0.024 KB total ukey len = 0.003209 GB avg = 0.016 KB total ukey num = 0.000200577 Billion total entry num = 0.000200577 Billion write speed all = 0.003795484 MB/sec (with seq num) write speed all = 0.003750559 MB/sec (without seq num) 2024/11/20-18:42:59.256743 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58826: 200577 keys, 130649247 bytes, temperature: kWarm 2024/11/20-18:42:59.256815 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099379256779, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58826, "file_size": 130649247, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985133052, "table_properties": {"data_size": 129616928, "index_size": 348800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4813848, "raw_average_key_size": 24, "raw_value_size": 129404129, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200577, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732099376, "oldest_key_time": 0, "file_creation_time": 1732099376, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58826, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:42:59.312673 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058827.sst) /dev/shm/ToplingTemp/Topling-847461-aBhIMc 2024/11/20-18:43:00.735069 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058827.sst): first pass time = 1.42's, 94.363'MB/sec 2024/11/20-18:43:02.091805 140008543020608 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058827.sst): part-0: reject ZipStore ratio = 0.801 2024/11/20-18:43:02.506597 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058827.sst): old prealloc_size = 147639500, real_size = 130015568 2024/11/20-18:43:02.506797 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058827.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200491 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.73 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.45 avg-zval = 646.51 2024/11/20-18:43:02.507008 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058827.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:43:02.916530 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058827.sst): second pass time = 1.36's, 95.448'MB/sec, value only(96.4% of KV) wait indexing time = 0.42's, remap KeyValue time = 0.41's, 318.334'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1953.186'MB/sec (index lex order gen) rebuild zvType time = 0.34's, 0.147'MB/sec write SST data time = 0.07's, 1925.958'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.36's, unzip length = 0.129'GB zip my value throughput = 95.448'MB/sec zip pipeline throughput = 117.697'MB/sec entries = 200491 avg-key = 24.00 avg-zkey = 1.73 avg-val = 645.45 avg-zval = 646.51 usrkeys = 200491 avg-key = 16.00 avg-zkey = 1.73 avg-val = 650.59 avg-zval = 646.51 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 147415 bytes = 497536 } seq expand size = 1031905 multi value expand size = 0 cnt WriteAppend = 15806 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1305 GB } UnZip/Zip{ index = 9.2700 value = 0.9984 dict = 0.00 all = 1.0285 } Zip/UnZip{ index = 0.1079 value = 1.0016 dict = 0.00 all = 0.9723 } ---------------------------- total value len = 0.261189 GB avg = 0.651 KB (by entry num) total key len = 0.009626 GB avg = 0.024 KB total ukey len = 0.006417 GB avg = 0.016 KB total ukey num = 0.000401068 Billion total entry num = 0.000401068 Billion write speed all = 0.007581361 MB/sec (with seq num) write speed all = 0.007491539 MB/sec (without seq num) 2024/11/20-18:43:03.061626 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58827: 200491 keys, 130497183 bytes, temperature: kWarm 2024/11/20-18:43:03.061685 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099383061656, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58827, "file_size": 130497183, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985132869, "table_properties": {"data_size": 129619200, "index_size": 346048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4811784, "raw_average_key_size": 24, "raw_value_size": 129406489, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200491, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732099379, "oldest_key_time": 0, "file_creation_time": 1732099379, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58827, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:43:03.120471 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058828.sst) /dev/shm/ToplingTemp/Topling-847461-BiJEBe 2024/11/20-18:43:04.371800 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058828.sst): first pass time = 1.25's, 107.263'MB/sec 2024/11/20-18:43:05.604090 140008543020608 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058828.sst): part-0: reject ZipStore ratio = 0.801 2024/11/20-18:43:05.863950 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058828.sst): old prealloc_size = 147639500, real_size = 130012736 2024/11/20-18:43:05.864122 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058828.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200454 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.71 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.57 avg-zval = 646.63 2024/11/20-18:43:05.864330 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058828.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:43:06.171313 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058828.sst): second pass time = 1.23's, 105.080'MB/sec, value only(96.4% of KV) wait indexing time = 0.26's, remap KeyValue time = 0.31's, 424.763'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2225.032'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.203'MB/sec write SST data time = 0.06's, 2156.384'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.23's, unzip length = 0.129'GB zip my value throughput = 105.080'MB/sec zip pipeline throughput = 116.328'MB/sec entries = 200454 avg-key = 24.00 avg-zkey = 1.71 avg-val = 645.57 avg-zval = 646.63 usrkeys = 200454 avg-key = 16.00 avg-zkey = 1.71 avg-val = 651.30 avg-zval = 646.63 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 164085 bytes = 553808 } seq expand size = 1148595 multi value expand size = 0 cnt WriteAppend = 15807 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1306 GB } UnZip/Zip{ index = 9.3565 value = 0.9984 dict = 0.00 all = 1.0281 } Zip/UnZip{ index = 0.1069 value = 1.0016 dict = 0.00 all = 0.9727 } ---------------------------- total value len = 0.391744 GB avg = 0.651 KB (by entry num) total key len = 0.014437 GB avg = 0.024 KB total ukey len = 0.009624 GB avg = 0.016 KB total ukey num = 0.000601522 Billion total entry num = 0.000601522 Billion write speed all = 0.011369865 MB/sec (with seq num) write speed all = 0.011235162 MB/sec (without seq num) 2024/11/20-18:43:06.333474 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58828: 200454 keys, 130550623 bytes, temperature: kWarm 2024/11/20-18:43:06.333557 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099386333512, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58828, "file_size": 130550623, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985132776, "table_properties": {"data_size": 129619632, "index_size": 342784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4810896, "raw_average_key_size": 24, "raw_value_size": 129406973, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200454, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732099383, "oldest_key_time": 0, "file_creation_time": 1732099383, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58828, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:43:06.415508 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058829.sst) /dev/shm/ToplingTemp/Topling-847461-ZxET5c 2024/11/20-18:43:07.765270 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058829.sst): first pass time = 1.35's, 99.444'MB/sec 2024/11/20-18:43:08.978962 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058829.sst): old prealloc_size = 147639500, real_size = 103858320 2024/11/20-18:43:08.979133 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058829.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200538 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.1294 GB zip-val = 0.1035 GB avg-val = 645.29 avg-zval = 515.95 2024/11/20-18:43:08.979278 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058829.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:43:09.255545 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058829.sst): second pass time = 1.21's, 106.689'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.28's, 381.659'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2272.106'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.234'MB/sec write SST data time = 0.06's, 1707.482'MB/sec dict compress time = 0.00's, 1349854.000'MB/sec z-dict build time = 0.00's, sample length = 1.350'MB, throughput = 43020.493'MB/sec zip my value time = 1.21's, unzip length = 0.129'GB zip my value throughput = 106.689'MB/sec zip pipeline throughput = 124.790'MB/sec entries = 200538 avg-key = 24.00 avg-zkey = 1.70 avg-val = 645.29 avg-zval = 515.95 usrkeys = 200538 avg-key = 16.00 avg-zkey = 1.70 avg-val = 649.17 avg-zval = 515.95 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 111031 bytes = 360864 } seq expand size = 777217 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.35 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1035 GB dict = 1.35 MB all = 0.1056 GB } UnZip/Zip{ index = 9.4149 value = 1.2507 dict = 1.00 all = 1.2716 } Zip/UnZip{ index = 0.1062 value = 0.7996 dict = 1.00 all = 0.7864 } ---------------------------- total value len = 0.521927 GB avg = 0.651 KB (by entry num) total key len = 0.019249 GB avg = 0.024 KB total ukey len = 0.012833 GB avg = 0.016 KB total ukey num = 0.000802060 Billion total entry num = 0.000802060 Billion write speed all = 0.015147364 MB/sec (with seq num) write speed all = 0.014967769 MB/sec (without seq num) 2024/11/20-18:43:09.371597 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58829: 200538 keys, 105553125 bytes, temperature: kWarm 2024/11/20-18:43:09.371666 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099389371635, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58829, "file_size": 105553125, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985129171, "table_properties": {"data_size": 103467200, "index_size": 340800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4812912, "raw_average_key_size": 24, "raw_value_size": 129405222, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200538, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732099386, "oldest_key_time": 0, "file_creation_time": 1732099386, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58829, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:43:09.414503 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058830.sst) /dev/shm/ToplingTemp/Topling-847461-RVm3Bd 2024/11/20-18:43:10.641333 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058830.sst): first pass time = 1.23's, 109.405'MB/sec 2024/11/20-18:43:11.623279 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058830.sst): old prealloc_size = 147639500, real_size = 103824432 2024/11/20-18:43:11.623423 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058830.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200609 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1034 GB avg-val = 645.05 avg-zval = 515.61 2024/11/20-18:43:11.623541 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058830.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:43:11.962783 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058830.sst): second pass time = 0.98's, 131.881'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.34's, 310.680'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2779.275'MB/sec (index lex order gen) rebuild zvType time = 0.26's, 0.196'MB/sec write SST data time = 0.08's, 1259.342'MB/sec dict compress time = 0.00's, 1344755.000'MB/sec z-dict build time = 0.00's, sample length = 1.345'MB, throughput = 42154.008'MB/sec zip my value time = 0.98's, unzip length = 0.129'GB zip my value throughput = 131.881'MB/sec zip pipeline throughput = 131.881'MB/sec entries = 200609 avg-key = 24.00 avg-zkey = 1.69 avg-val = 645.05 avg-zval = 515.61 usrkeys = 200609 avg-key = 16.00 avg-zkey = 1.69 avg-val = 648.38 avg-zval = 515.61 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 95215 bytes = 309456 } seq expand size = 666505 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.34 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1034 GB dict = 1.34 MB all = 0.1055 GB } UnZip/Zip{ index = 9.4770 value = 1.2511 dict = 1.00 all = 1.2727 } Zip/UnZip{ index = 0.1055 value = 0.7993 dict = 1.00 all = 0.7858 } ---------------------------- total value len = 0.651997 GB avg = 0.650 KB (by entry num) total key len = 0.024064 GB avg = 0.024 KB total ukey len = 0.016043 GB avg = 0.016 KB total ukey num = 0.001002669 Billion total entry num = 0.001002669 Billion write speed all = 0.018921311 MB/sec (with seq num) write speed all = 0.018696812 MB/sec (without seq num) 2024/11/20-18:43:12.085507 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58830: 200609 keys, 105462778 bytes, temperature: kWarm 2024/11/20-18:43:12.085577 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099392085546, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58830, "file_size": 105462778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985116844, "table_properties": {"data_size": 103435424, "index_size": 338688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4814616, "raw_average_key_size": 24, "raw_value_size": 129403462, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200609, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732099389, "oldest_key_time": 0, "file_creation_time": 1732099389, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58830, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:43:12.126218 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058831.sst) /dev/shm/ToplingTemp/Topling-847461-Pa8yag 2024/11/20-18:43:13.573911 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058831.sst): first pass time = 1.45's, 92.714'MB/sec 2024/11/20-18:43:14.827417 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058831.sst): old prealloc_size = 147639500, real_size = 103819904 2024/11/20-18:43:14.827564 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058831.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200681 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1034 GB avg-val = 644.81 avg-zval = 515.40 2024/11/20-18:43:14.827629 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058831.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:43:15.219848 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058831.sst): second pass time = 1.25's, 103.292'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.39's, 267.992'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2688.956'MB/sec (index lex order gen) rebuild zvType time = 0.33's, 0.150'MB/sec write SST data time = 0.06's, 1798.052'MB/sec dict compress time = 0.00's, 1345633.000'MB/sec z-dict build time = 0.00's, sample length = 1.346'MB, throughput = 44832.017'MB/sec zip my value time = 1.25's, unzip length = 0.129'GB zip my value throughput = 103.292'MB/sec zip pipeline throughput = 103.292'MB/sec entries = 200681 avg-key = 24.00 avg-zkey = 1.69 avg-val = 644.81 avg-zval = 515.40 usrkeys = 200681 avg-key = 16.00 avg-zkey = 1.69 avg-val = 644.96 avg-zval = 515.40 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 4242 bytes = 13808 } seq expand size = 29694 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.35 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1034 GB dict = 1.35 MB all = 0.1052 GB } UnZip/Zip{ index = 9.4607 value = 1.2511 dict = 1.00 all = 1.2763 } Zip/UnZip{ index = 0.1057 value = 0.7993 dict = 1.00 all = 0.7835 } ---------------------------- total value len = 0.781428 GB avg = 0.649 KB (by entry num) total key len = 0.028880 GB avg = 0.024 KB total ukey len = 0.019254 GB avg = 0.016 KB total ukey num = 0.001203350 Billion total entry num = 0.001203350 Billion write speed all = 0.022676517 MB/sec (with seq num) write speed all = 0.022407110 MB/sec (without seq num) 2024/11/20-18:43:15.424027 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58831: 200681 keys, 105163432 bytes, temperature: kWarm 2024/11/20-18:43:15.424106 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099395424075, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58831, "file_size": 105163432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985124923, "table_properties": {"data_size": 103430128, "index_size": 339392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4816344, "raw_average_key_size": 24, "raw_value_size": 129401806, "raw_average_value_size": 644, "num_data_blocks": 1, "num_entries": 200681, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732099392, "oldest_key_time": 0, "file_creation_time": 1732099392, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58831, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:43:15.467017 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058832.sst) /dev/shm/ToplingTemp/Topling-847461-pNIHjf 2024/11/20-18:43:16.673580 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058832.sst): first pass time = 1.21's, 111.246'MB/sec 2024/11/20-18:43:18.061370 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058832.sst): old prealloc_size = 147639500, real_size = 103847904 2024/11/20-18:43:18.061534 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058832.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200525 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1035 GB avg-val = 645.33 avg-zval = 515.94 2024/11/20-18:43:18.061635 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058832.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:43:18.441799 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058832.sst): second pass time = 1.39's, 93.309'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.38's, 276.885'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2577.850'MB/sec (index lex order gen) rebuild zvType time = 0.32's, 0.156'MB/sec write SST data time = 0.06's, 1776.505'MB/sec dict compress time = 0.00's, 1349676.000'MB/sec z-dict build time = 0.00's, sample length = 1.350'MB, throughput = 23156.092'MB/sec zip my value time = 1.39's, unzip length = 0.129'GB zip my value throughput = 93.309'MB/sec zip pipeline throughput = 191.392'MB/sec entries = 200525 avg-key = 24.00 avg-zkey = 1.69 avg-val = 645.33 avg-zval = 515.94 usrkeys = 200525 avg-key = 16.00 avg-zkey = 1.69 avg-val = 646.98 avg-zval = 515.94 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 47064 bytes = 152976 } seq expand size = 329448 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.35 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1035 GB dict = 1.35 MB all = 0.1053 GB } UnZip/Zip{ index = 9.4534 value = 1.2508 dict = 1.00 all = 1.2742 } Zip/UnZip{ index = 0.1058 value = 0.7995 dict = 1.00 all = 0.7848 } ---------------------------- total value len = 0.911163 GB avg = 0.649 KB (by entry num) total key len = 0.033693 GB avg = 0.024 KB total ukey len = 0.022462 GB avg = 0.016 KB total ukey num = 0.001403875 Billion total entry num = 0.001403875 Billion write speed all = 0.026439452 MB/sec (with seq num) write speed all = 0.026125180 MB/sec (without seq num) 2024/11/20-18:43:18.556744 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58832: 200525 keys, 105334643 bytes, temperature: kWarm 2024/11/20-18:43:18.556813 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099398556782, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58832, "file_size": 105334643, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985126974, "table_properties": {"data_size": 103458192, "index_size": 339392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4812600, "raw_average_key_size": 24, "raw_value_size": 129405532, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200525, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732099395, "oldest_key_time": 0, "file_creation_time": 1732099395, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58832, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:43:18.600315 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058833.sst) /dev/shm/ToplingTemp/Topling-847461-AsBqle 2024/11/20-18:43:20.030550 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058833.sst): first pass time = 1.43's, 93.846'MB/sec 2024/11/20-18:43:21.260723 140008543020608 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058833.sst): part-0: reject ZipStore ratio = 0.801 2024/11/20-18:43:21.620909 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058833.sst): old prealloc_size = 147639500, real_size = 130008016 2024/11/20-18:43:21.621086 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058833.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200500 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.42 avg-zval = 646.48 2024/11/20-18:43:21.621170 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058833.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:43:21.927220 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058833.sst): second pass time = 1.23's, 105.270'MB/sec, value only(96.4% of KV) wait indexing time = 0.36's, remap KeyValue time = 0.31's, 424.698'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2119.036'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.199'MB/sec write SST data time = 0.05's, 2406.579'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.23's, unzip length = 0.129'GB zip my value throughput = 105.270'MB/sec zip pipeline throughput = 105.270'MB/sec entries = 200500 avg-key = 24.00 avg-zkey = 1.69 avg-val = 645.42 avg-zval = 646.48 usrkeys = 200500 avg-key = 16.00 avg-zkey = 1.69 avg-val = 646.46 avg-zval = 646.48 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 29980 bytes = 93696 } seq expand size = 209860 multi value expand size = 0 cnt WriteAppend = 15806 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1301 GB } UnZip/Zip{ index = 9.4736 value = 0.9984 dict = 0.00 all = 1.0318 } Zip/UnZip{ index = 0.1056 value = 1.0016 dict = 0.00 all = 0.9692 } ---------------------------- total value len = 1.040779 GB avg = 0.649 KB (by entry num) total key len = 0.038505 GB avg = 0.024 KB total ukey len = 0.025670 GB avg = 0.016 KB total ukey num = 0.001604375 Billion total entry num = 0.001604375 Billion write speed all = 0.030198146 MB/sec (with seq num) write speed all = 0.029839026 MB/sec (without seq num) 2024/11/20-18:43:22.074576 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58833: 200500 keys, 130085791 bytes, temperature: kWarm 2024/11/20-18:43:22.074636 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099402074606, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58833, "file_size": 130085791, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985132350, "table_properties": {"data_size": 129619072, "index_size": 338624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4812000, "raw_average_key_size": 24, "raw_value_size": 129406359, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200500, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732099398, "oldest_key_time": 0, "file_creation_time": 1732099398, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58833, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:43:22.130754 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058834.sst) /dev/shm/ToplingTemp/Topling-847461-hMH9Tb 2024/11/20-18:43:23.403527 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058834.sst): first pass time = 1.27's, 105.457'MB/sec 2024/11/20-18:43:24.652967 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058834.sst): old prealloc_size = 147639500, real_size = 103841728 2024/11/20-18:43:24.653201 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058834.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200611 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1035 GB avg-val = 645.05 avg-zval = 515.68 2024/11/20-18:43:24.653382 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058834.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:43:25.003486 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058834.sst): second pass time = 1.25's, 103.644'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.35's, 301.260'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1611.798'MB/sec (index lex order gen) rebuild zvType time = 0.28's, 0.177'MB/sec write SST data time = 0.07's, 1579.016'MB/sec dict compress time = 0.00's, 1349385.000'MB/sec z-dict build time = 0.00's, sample length = 1.349'MB, throughput = 40529.375'MB/sec zip my value time = 1.25's, unzip length = 0.129'GB zip my value throughput = 103.644'MB/sec zip pipeline throughput = 115.997'MB/sec entries = 200611 avg-key = 24.00 avg-zkey = 1.69 avg-val = 645.05 avg-zval = 515.68 usrkeys = 200611 avg-key = 16.00 avg-zkey = 1.69 avg-val = 649.72 avg-zval = 515.68 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 133997 bytes = 418752 } seq expand size = 937979 multi value expand size = 0 cnt WriteAppend = 23 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.35 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1035 GB dict = 1.35 MB all = 0.1056 GB } UnZip/Zip{ index = 9.4432 value = 1.2509 dict = 1.00 all = 1.2711 } Zip/UnZip{ index = 0.1059 value = 0.7994 dict = 1.00 all = 0.7867 } ---------------------------- total value len = 1.171121 GB avg = 0.649 KB (by entry num) total key len = 0.043320 GB avg = 0.024 KB total ukey len = 0.028880 GB avg = 0.016 KB total ukey num = 0.001804986 Billion total entry num = 0.001804986 Billion write speed all = 0.033976868 MB/sec (with seq num) write speed all = 0.033572878 MB/sec (without seq num) 2024/11/20-18:43:25.120784 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58834: 200611 keys, 105594192 bytes, temperature: kWarm 2024/11/20-18:43:25.120845 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099405120815, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58834, "file_size": 105594192, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985124816, "table_properties": {"data_size": 103451504, "index_size": 339904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4814664, "raw_average_key_size": 24, "raw_value_size": 129403737, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200611, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732099402, "oldest_key_time": 0, "file_creation_time": 1732099402, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58834, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:43:25.162244 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058835.sst) /dev/shm/ToplingTemp/Topling-847461-mWMlKc 2024/11/20-18:43:26.315275 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058835.sst): first pass time = 1.15's, 116.408'MB/sec 2024/11/20-18:43:27.478042 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058835.sst): old prealloc_size = 147639500, real_size = 103842128 2024/11/20-18:43:27.478214 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058835.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200565 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1035 GB avg-val = 645.20 avg-zval = 515.80 2024/11/20-18:43:27.478294 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058835.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-18:43:27.798952 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058835.sst): second pass time = 1.16's, 111.360'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 328.146'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2586.883'MB/sec (index lex order gen) rebuild zvType time = 0.24's, 0.213'MB/sec write SST data time = 0.09's, 1234.681'MB/sec dict compress time = 0.00's, 1348235.000'MB/sec z-dict build time = 0.00's, sample length = 1.348'MB, throughput = 21944.286'MB/sec zip my value time = 1.16's, unzip length = 0.129'GB zip my value throughput = 111.360'MB/sec zip pipeline throughput = 111.360'MB/sec entries = 200565 avg-key = 24.00 avg-zkey = 1.69 avg-val = 645.20 avg-zval = 515.80 usrkeys = 200565 avg-key = 16.00 avg-zkey = 1.69 avg-val = 646.50 avg-zval = 515.80 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 37122 bytes = 116016 } seq expand size = 259854 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.35 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1035 GB dict = 1.35 MB all = 0.1053 GB } UnZip/Zip{ index = 9.4535 value = 1.2509 dict = 1.00 all = 1.2747 } Zip/UnZip{ index = 0.1058 value = 0.7994 dict = 1.00 all = 0.7845 } ---------------------------- total value len = 1.300786 GB avg = 0.649 KB (by entry num) total key len = 0.048133 GB avg = 0.024 KB total ukey len = 0.032089 GB avg = 0.016 KB total ukey num = 0.002005551 Billion total entry num = 0.002005551 Billion write speed all = 0.037736264 MB/sec (with seq num) write speed all = 0.037287419 MB/sec (without seq num) 2024/11/20-18:43:27.921326 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58835: 200565 keys, 105290514 bytes, temperature: kWarm 2024/11/20-18:43:27.921392 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099407921360, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58835, "file_size": 105290514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985132312, "table_properties": {"data_size": 103452352, "index_size": 339456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4813560, "raw_average_key_size": 24, "raw_value_size": 129404791, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200565, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732099405, "oldest_key_time": 0, "file_creation_time": 1732099405, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58835, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:43:27.962421 140008543020608 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058836.sst) /dev/shm/ToplingTemp/Topling-847461-jdpQmf 2024/11/20-18:43:28.843391 140008543020608 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058836.sst): first pass time = 0.88's, 96.479'MB/sec 2024/11/20-18:43:29.617725 140008543020608 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058836.sst): part-0: reject ZipStore ratio = 0.802 2024/11/20-18:43:29.791809 140008543020608 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058836.sst): old prealloc_size = 147639500, real_size = 82326240 2024/11/20-18:43:29.791919 140008543020608 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058836.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 127034 prefix = 4 raw-key = 0.0015 GB zip-key = 0.0002 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.0819 GB zip-val = 0.0821 GB avg-val = 645.05 avg-zval = 646.11 2024/11/20-18:43:29.792035 140008543020608 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058836.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.085 GB 2024/11/20-18:43:29.984628 140008543020608 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00/058836.sst): second pass time = 0.77's, 105.915'MB/sec, value only(96.4% of KV) wait indexing time = 0.17's, remap KeyValue time = 0.19's, 428.594'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2280.562'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.201'MB/sec write SST data time = 0.03's, 2391.603'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.77's, unzip length = 0.082'GB zip my value throughput = 105.915'MB/sec zip pipeline throughput = 134.162'MB/sec entries = 127034 avg-key = 24.00 avg-zkey = 1.70 avg-val = 645.05 avg-zval = 646.11 usrkeys = 127034 avg-key = 16.00 avg-zkey = 1.70 avg-val = 650.99 avg-zval = 646.11 TagRS{ kinds = RS_Key0_TagN bytes = 19960 } TagArray{ size = 107780 bytes = 323360 } seq expand size = 754460 multi value expand size = 0 cnt WriteAppend = 10014 UnZipSize{ index = 0.0020 GB value = 0.0819 GB dict = 0.00 MB all = 0.0850 GB } __ZipSize{ index = 0.0002 GB value = 0.0821 GB dict = 0.00 MB all = 0.0826 GB } UnZip/Zip{ index = 9.3877 value = 0.9984 dict = 0.00 all = 1.0285 } Zip/UnZip{ index = 0.1065 value = 1.0016 dict = 0.00 all = 0.9723 } ---------------------------- total value len = 1.383483 GB avg = 0.649 KB (by entry num) total key len = 0.051182 GB avg = 0.024 KB total ukey len = 0.034121 GB avg = 0.016 KB total ukey num = 0.002132585 Billion total entry num = 0.002132585 Billion write speed all = 0.040132579 MB/sec (with seq num) write speed all = 0.039655332 MB/sec (without seq num) 2024/11/20-18:43:30.078491 140008543020608 [compaction_job.cc:1948] [default] [JOB 6591] Generated table #58836: 127034 keys, 82640031 bytes, temperature: kWarm 2024/11/20-18:43:30.078559 140008543020608 EVENT_LOG_v1 {"time_micros": 1732099410078527, "cf_name": "default", "job": 6591, "event": "table_file_creation", "file_number": 58836, "file_size": 82640031, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 985129139, "table_properties": {"data_size": 82077776, "index_size": 216512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3048816, "raw_average_key_size": 24, "raw_value_size": 81943010, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 127034, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732099407, "oldest_key_time": 0, "file_creation_time": 1732099407, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 58836, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-18:43:30.108416 140008543020608 [compaction_job.cc:703] job-06591: subcompact[0], size: 1.236822 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-06591/att-00 [058826,058827,058828,058829,058830,058831,058832,058833,058834,058835,058836] 2024/11/20-18:43:30.294342 140008543020608 (Original Log Time 2024/11/20-18:42:55.412135) EVENT_LOG_v1 {"time_micros": 1732099375412103, "job": 6591, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [58820], "files_L3": [58735, 58736, 58737, 58738, 58739, 58740, 58741, 58742, 58743, 58744, 58745], "score": 0.780632, "input_data_size": 1253998862, "oldest_snapshot_seqno": 985179858} 2024/11/20-18:43:30.294347 140008543020608 (Original Log Time 2024/11/20-18:43:30.292895) [compaction_job.cc:2022] [default] [JOB 6591] Compacted 1@2 + 11@3 files to L3 => 1236821559 bytes 2024/11/20-18:43:30.294350 140008543020608 (Original Log Time 2024/11/20-18:43:30.294243) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.05, MB/sec: 36.1 rd, 35.6 wr, level 3, files in(1, 11) out(11 +0 blob) MB in(22.0, 1173.9 +0.0 blob) out(1179.5 +0.0 blob), read-write-amplify(108.0) write-amplify(53.6) OK, records in: 2158734, records dropped: 26149 output_compression: Snappy 2024/11/20-18:43:30.294357 140008543020608 (Original Log Time 2024/11/20-18:43:30.294313) EVENT_LOG_v1 {"time_micros": 1732099410294276, "cf": "default", "job": 6591, "event": "compaction_finished", "compaction_time_micros": 34696289, "compaction_time_cpu_micros": 7887875, "output_level": 3, "num_output_files": 11, "total_output_size": 1236821559, "num_input_records": 2158734, "num_output_records": 2132585, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 957730667, "file_range_sync_nanos": 0, "file_fsync_nanos": 10280, "file_prepare_write_nanos": 3231102, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/20-18:43:30.451393 140008543020608 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732099375,"executesMs":34998,"compactionJobId":6591,"attempt":0,"compactionInputRawBytes":1444775276,"compactionInputZipBytes":1253998862,"compactionOutputRawBytes":0,"compactionOutputZipBytes":1236821559,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-1b43cb74784a27b58b7cdaa7adca0aba-ed499b802852b1ce-00"}}, response = 2024/11/20-18:43:30.451459 140008543020608 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-06591/att-00: olev 3, work 34.998 s, result 8.798 ms, install 1.486 ms, input{raw 1.346 GiB zip 1.168 GiB}