2024/11/20-19:26:54.180635 140008778016320 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00: fork to child time = 0.010078 sec 2024/11/20-19:26:54.181161 140008778016320 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00 2024/11/20-19:26:54.193823 140008778016320 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00 2024/11/20-19:26:54.195519 140008778016320 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-19:26:54.195765 140008778016320 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-19:26:54.243521 140008778016320 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-08083/att-00/MANIFEST-049091 2024/11/20-19:26:54.293656 140008778016320 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-08083/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 61778, last_sequence is 1020021386, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 61626 2024/11/20-19:26:54.293664 140008778016320 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-19:26:54.293737 140008778016320 [version_set.cc:5724] Creating manifest 61778 2024/11/20-19:26:54.326207 140008778016320 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-19:26:54.349322 140008778016320 [compaction_job.cc:2437] [default] [JOB 8083] Compacting 1@2 + 11@3 files to L3, score 0.98, subcompactions 1 : 1 2024/11/20-19:26:54.349336 140008778016320 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 7253 Base level 2, inputs: [61773(21MB)], [61706(124MB) 61707(124MB) 61708(124MB) 61709(124MB) 61710(100MB) 61711(100MB) 61712(100MB) 61713(124MB) 61714(124MB) 61717(124MB) 61718(21MB)] 2024/11/20-19:26:54.356086 140008778016320 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-19:26:54.356110 140008778016320 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-19:26:54.356129 140008778016320 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-19:26:54.356137 140008778016320 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-19:26:54.356147 140008778016320 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-19:26:55.598915 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061779.sst) /dev/shm/ToplingTemp/Topling-886089-Nq0Dj7 2024/11/20-19:26:55.762842 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061779.sst): first pass time = 0.16's, 819.014'MB/sec 2024/11/20-19:26:57.280076 140008778016320 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061779.sst): part-0: reject ZipStore ratio = 0.803 2024/11/20-19:26:57.554191 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061779.sst): old prealloc_size = 147639500, real_size = 130030160 2024/11/20-19:26:57.554356 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061779.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200234 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0004 GB avg-key = 12.00 avg-zkey = 1.77 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 646.31 avg-zval = 647.37 2024/11/20-19:26:57.554599 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061779.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:26:57.862753 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061779.sst): second pass time = 1.52's, 85.342'MB/sec, value only(96.4% of KV) wait indexing time = 0.27's, remap KeyValue time = 0.31's, 423.544'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2389.974'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.199'MB/sec write SST data time = 0.06's, 2307.318'MB/sec dict compress time = 3472328296.23's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.52's, unzip length = 0.129'GB zip my value throughput = 85.342'MB/sec zip pipeline throughput = 181.281'MB/sec entries = 200234 avg-key = 24.00 avg-zkey = 1.77 avg-val = 646.31 avg-zval = 647.37 usrkeys = 200234 avg-key = 16.00 avg-zkey = 1.77 avg-val = 653.02 avg-zval = 647.37 TagRS{ kinds = RS_Key0_TagN bytes = 31400 } TagArray{ size = 192047 bytes = 672176 } seq expand size = 1344329 multi value expand size = 0 cnt WriteAppend = 15808 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0004 GB value = 0.1296 GB dict = 0.00 MB all = 0.1307 GB } UnZip/Zip{ index = 9.0277 value = 0.9984 dict = 0.00 all = 1.0270 } Zip/UnZip{ index = 0.1108 value = 1.0016 dict = 0.00 all = 0.9737 } ---------------------------- total value len = 0.130757 GB avg = 0.653 KB (by entry num) total key len = 0.004806 GB avg = 0.024 KB total ukey len = 0.003204 GB avg = 0.016 KB total ukey num = 0.000200234 Billion total entry num = 0.000200234 Billion write speed all = 0.003534322 MB/sec (with seq num) write speed all = 0.003492559 MB/sec (without seq num) 2024/11/20-19:26:58.030795 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61779: 200234 keys, 130686431 bytes, temperature: kWarm 2024/11/20-19:26:58.030877 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102018030836, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61779, "file_size": 130686431, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019989685, "table_properties": {"data_size": 129625024, "index_size": 354880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4805616, "raw_average_key_size": 24, "raw_value_size": 129412591, "raw_average_value_size": 646, "num_data_blocks": 1, "num_entries": 200234, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102015, "oldest_key_time": 0, "file_creation_time": 1732102015, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61779, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:26:58.091994 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061780.sst) /dev/shm/ToplingTemp/Topling-886089-zZc325 2024/11/20-19:26:59.470780 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061780.sst): first pass time = 1.38's, 97.347'MB/sec 2024/11/20-19:27:00.854659 140008778016320 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061780.sst): part-0: reject ZipStore ratio = 0.802 2024/11/20-19:27:01.134657 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061780.sst): old prealloc_size = 147639500, real_size = 130022480 2024/11/20-19:27:01.134855 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061780.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200386 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0004 GB avg-key = 12.00 avg-zkey = 1.75 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.80 avg-zval = 646.86 2024/11/20-19:27:01.135080 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061780.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:27:01.529584 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061780.sst): second pass time = 1.38's, 93.567'MB/sec, value only(96.4% of KV) wait indexing time = 0.28's, remap KeyValue time = 0.39's, 330.511'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2011.384'MB/sec (index lex order gen) rebuild zvType time = 0.34's, 0.149'MB/sec write SST data time = 0.06's, 2249.439'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.38's, unzip length = 0.129'GB zip my value throughput = 93.567'MB/sec zip pipeline throughput = 94.897'MB/sec entries = 200386 avg-key = 24.00 avg-zkey = 1.75 avg-val = 645.80 avg-zval = 646.86 usrkeys = 200386 avg-key = 16.00 avg-zkey = 1.75 avg-val = 650.96 avg-zval = 646.86 TagRS{ kinds = RS_Key0_TagN bytes = 31400 } TagArray{ size = 147924 bytes = 517744 } seq expand size = 1035468 multi value expand size = 0 cnt WriteAppend = 15807 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0004 GB value = 0.1296 GB dict = 0.00 MB all = 0.1305 GB } UnZip/Zip{ index = 9.1367 value = 0.9984 dict = 0.00 all = 1.0283 } Zip/UnZip{ index = 0.1094 value = 1.0016 dict = 0.00 all = 0.9725 } ---------------------------- total value len = 0.261201 GB avg = 0.652 KB (by entry num) total key len = 0.009615 GB avg = 0.024 KB total ukey len = 0.006410 GB avg = 0.016 KB total ukey num = 0.000400620 Billion total entry num = 0.000400620 Billion write speed all = 0.007059909 MB/sec (with seq num) write speed all = 0.006976359 MB/sec (without seq num) 2024/11/20-19:27:01.737974 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61780: 200386 keys, 130524319 bytes, temperature: kWarm 2024/11/20-19:27:01.738032 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102021738001, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61780, "file_size": 130524319, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019989601, "table_properties": {"data_size": 129621312, "index_size": 350912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4809264, "raw_average_key_size": 24, "raw_value_size": 129408655, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200386, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102018, "oldest_key_time": 0, "file_creation_time": 1732102018, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61780, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:27:01.797379 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061781.sst) /dev/shm/ToplingTemp/Topling-886089-bKJUY6 2024/11/20-19:27:03.201701 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061781.sst): first pass time = 1.40's, 95.578'MB/sec 2024/11/20-19:27:04.468146 140008778016320 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061781.sst): part-0: reject ZipStore ratio = 0.801 2024/11/20-19:27:04.787648 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061781.sst): old prealloc_size = 147639500, real_size = 130025696 2024/11/20-19:27:04.787805 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061781.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200009 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.73 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 647.06 avg-zval = 648.12 2024/11/20-19:27:04.787999 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061781.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:27:05.105148 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061781.sst): second pass time = 1.27's, 102.263'MB/sec, value only(96.4% of KV) wait indexing time = 0.32's, remap KeyValue time = 0.32's, 411.300'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2470.350'MB/sec (index lex order gen) rebuild zvType time = 0.26's, 0.194'MB/sec write SST data time = 0.06's, 2195.277'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.27's, unzip length = 0.129'GB zip my value throughput = 102.263'MB/sec zip pipeline throughput = 133.179'MB/sec entries = 200009 avg-key = 24.00 avg-zkey = 1.73 avg-val = 647.06 avg-zval = 648.12 usrkeys = 200009 avg-key = 16.00 avg-zkey = 1.73 avg-val = 653.03 avg-zval = 648.12 TagRS{ kinds = RS_Key0_TagN bytes = 31360 } TagArray{ size = 170673 bytes = 576032 } seq expand size = 1194711 multi value expand size = 0 cnt WriteAppend = 15809 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1306 GB } UnZip/Zip{ index = 9.2562 value = 0.9984 dict = 0.00 all = 1.0278 } Zip/UnZip{ index = 0.1080 value = 1.0016 dict = 0.00 all = 0.9729 } ---------------------------- total value len = 0.391813 GB avg = 0.652 KB (by entry num) total key len = 0.014415 GB avg = 0.024 KB total ukey len = 0.009610 GB avg = 0.016 KB total ukey num = 0.000600629 Billion total entry num = 0.000600629 Billion write speed all = 0.010588994 MB/sec (with seq num) write speed all = 0.010463743 MB/sec (without seq num) 2024/11/20-19:27:05.273103 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61781: 200009 keys, 130585887 bytes, temperature: kWarm 2024/11/20-19:27:05.273174 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102025273142, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61781, "file_size": 130585887, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019989862, "table_properties": {"data_size": 129629776, "index_size": 345728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4800216, "raw_average_key_size": 24, "raw_value_size": 129417533, "raw_average_value_size": 647, "num_data_blocks": 1, "num_entries": 200009, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102021, "oldest_key_time": 0, "file_creation_time": 1732102021, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61781, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:27:05.329455 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061782.sst) /dev/shm/ToplingTemp/Topling-886089-HWOhB3 2024/11/20-19:27:06.833752 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061782.sst): first pass time = 1.50's, 89.227'MB/sec 2024/11/20-19:27:07.753632 140008778016320 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061782.sst): part-0: reject ZipStore ratio = 0.801 2024/11/20-19:27:08.016517 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061782.sst): old prealloc_size = 147639500, real_size = 130014224 2024/11/20-19:27:08.016683 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061782.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200433 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.71 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.64 avg-zval = 646.70 2024/11/20-19:27:08.016826 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061782.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:27:08.325541 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061782.sst): second pass time = 0.92's, 140.802'MB/sec, value only(96.4% of KV) wait indexing time = 0.26's, remap KeyValue time = 0.31's, 421.847'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2341.208'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.202'MB/sec write SST data time = 0.06's, 2136.027'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.92's, unzip length = 0.129'GB zip my value throughput = 140.802'MB/sec zip pipeline throughput = 140.802'MB/sec entries = 200433 avg-key = 24.00 avg-zkey = 1.71 avg-val = 645.64 avg-zval = 646.70 usrkeys = 200433 avg-key = 16.00 avg-zkey = 1.71 avg-val = 649.32 avg-zval = 646.70 TagRS{ kinds = RS_Key0_TagN bytes = 31400 } TagArray{ size = 105369 bytes = 355632 } seq expand size = 737583 multi value expand size = 0 cnt WriteAppend = 15807 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1304 GB } UnZip/Zip{ index = 9.3311 value = 0.9984 dict = 0.00 all = 1.0296 } Zip/UnZip{ index = 0.1072 value = 1.0016 dict = 0.00 all = 0.9712 } ---------------------------- total value len = 0.521958 GB avg = 0.652 KB (by entry num) total key len = 0.019225 GB avg = 0.024 KB total ukey len = 0.012817 GB avg = 0.016 KB total ukey num = 0.000801062 Billion total entry num = 0.000801062 Billion write speed all = 0.014105642 MB/sec (with seq num) write speed all = 0.013938608 MB/sec (without seq num) 2024/11/20-19:27:08.483795 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61782: 200433 keys, 130353887 bytes, temperature: kWarm 2024/11/20-19:27:08.483863 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102028483832, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61782, "file_size": 130353887, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019985183, "table_properties": {"data_size": 129620224, "index_size": 343680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4810392, "raw_average_key_size": 24, "raw_value_size": 129407578, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102025, "oldest_key_time": 0, "file_creation_time": 1732102025, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61782, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:27:08.540157 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061783.sst) /dev/shm/ToplingTemp/Topling-886089-Elda26 2024/11/20-19:27:09.829276 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061783.sst): first pass time = 1.29's, 104.120'MB/sec 2024/11/20-19:27:10.995727 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061783.sst): old prealloc_size = 147639500, real_size = 103897872 2024/11/20-19:27:10.995877 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061783.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200531 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1035 GB avg-val = 645.31 avg-zval = 516.17 2024/11/20-19:27:10.996038 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061783.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:27:11.271687 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061783.sst): second pass time = 1.17's, 111.010'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.28's, 383.046'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2694.255'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.235'MB/sec write SST data time = 0.06's, 1684.022'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.342'MB, throughput = 38908.237'MB/sec zip my value time = 1.17's, unzip length = 0.129'GB zip my value throughput = 111.010'MB/sec zip pipeline throughput = 111.010'MB/sec entries = 200531 avg-key = 24.00 avg-zkey = 1.69 avg-val = 645.31 avg-zval = 516.17 usrkeys = 200531 avg-key = 16.00 avg-zkey = 1.69 avg-val = 650.25 avg-zval = 516.17 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 141284 bytes = 476848 } seq expand size = 988988 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.34 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1035 GB dict = 1.34 MB all = 0.1057 GB } UnZip/Zip{ index = 9.4626 value = 1.2502 dict = 1.00 all = 1.2698 } Zip/UnZip{ index = 0.1057 value = 0.7999 dict = 1.00 all = 0.7875 } ---------------------------- total value len = 0.652353 GB avg = 0.651 KB (by entry num) total key len = 0.024038 GB avg = 0.024 KB total ukey len = 0.016025 GB avg = 0.016 KB total ukey num = 0.001001593 Billion total entry num = 0.001001593 Billion write speed all = 0.017628388 MB/sec (with seq num) write speed all = 0.017419557 MB/sec (without seq num) 2024/11/20-19:27:11.400956 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61783: 200531 keys, 105701267 bytes, temperature: kWarm 2024/11/20-19:27:11.401023 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102031400991, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61783, "file_size": 105701267, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019986998, "table_properties": {"data_size": 103508480, "index_size": 339072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4812744, "raw_average_key_size": 24, "raw_value_size": 129405601, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200531, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102028, "oldest_key_time": 0, "file_creation_time": 1732102028, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61783, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:27:11.445350 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061784.sst) /dev/shm/ToplingTemp/Topling-886089-V2DEp5 2024/11/20-19:27:12.935028 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061784.sst): first pass time = 1.49's, 90.102'MB/sec 2024/11/20-19:27:14.093776 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061784.sst): old prealloc_size = 147639500, real_size = 103858704 2024/11/20-19:27:14.093945 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061784.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200634 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1035 GB avg-val = 644.97 avg-zval = 515.71 2024/11/20-19:27:14.094073 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061784.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:27:14.371421 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061784.sst): second pass time = 1.16's, 111.763'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.28's, 379.512'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2659.695'MB/sec (index lex order gen) rebuild zvType time = 0.22's, 0.228'MB/sec write SST data time = 0.06's, 1842.500'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.346'MB, throughput = 41263.771'MB/sec zip my value time = 1.16's, unzip length = 0.129'GB zip my value throughput = 111.763'MB/sec zip pipeline throughput = 111.763'MB/sec entries = 200634 avg-key = 24.00 avg-zkey = 1.69 avg-val = 644.97 avg-zval = 515.71 usrkeys = 200634 avg-key = 16.00 avg-zkey = 1.69 avg-val = 646.85 avg-zval = 515.71 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 53775 bytes = 181504 } seq expand size = 376425 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.35 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1035 GB dict = 1.35 MB all = 0.1054 GB } UnZip/Zip{ index = 9.4478 value = 1.2506 dict = 1.00 all = 1.2738 } Zip/UnZip{ index = 0.1058 value = 0.7996 dict = 1.00 all = 0.7851 } ---------------------------- total value len = 0.782132 GB avg = 0.651 KB (by entry num) total key len = 0.028853 GB avg = 0.024 KB total ukey len = 0.019236 GB avg = 0.016 KB total ukey num = 0.001202227 Billion total entry num = 0.001202227 Billion write speed all = 0.021134537 MB/sec (with seq num) write speed all = 0.020883893 MB/sec (without seq num) 2024/11/20-19:27:14.510568 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61784: 200634 keys, 105370475 bytes, temperature: kWarm 2024/11/20-19:27:14.510634 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102034510603, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61784, "file_size": 105370475, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019980209, "table_properties": {"data_size": 103468608, "index_size": 339776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4815216, "raw_average_key_size": 24, "raw_value_size": 129402794, "raw_average_value_size": 644, "num_data_blocks": 1, "num_entries": 200634, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102031, "oldest_key_time": 0, "file_creation_time": 1732102031, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61784, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:27:14.553602 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061785.sst) /dev/shm/ToplingTemp/Topling-886089-7an6m4 2024/11/20-19:27:15.641456 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061785.sst): first pass time = 1.09's, 123.383'MB/sec 2024/11/20-19:27:16.722093 140008778016320 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061785.sst): part-0: reject ZipStore ratio = 0.800 2024/11/20-19:27:16.978795 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061785.sst): old prealloc_size = 147639500, real_size = 130007712 2024/11/20-19:27:16.978956 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061785.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200552 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.24 avg-zval = 646.30 2024/11/20-19:27:16.979029 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061785.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:27:17.280106 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061785.sst): second pass time = 1.08's, 119.835'MB/sec, value only(96.4% of KV) wait indexing time = 0.26's, remap KeyValue time = 0.30's, 431.657'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2248.815'MB/sec (index lex order gen) rebuild zvType time = 0.24's, 0.206'MB/sec write SST data time = 0.06's, 2249.827'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.08's, unzip length = 0.129'GB zip my value throughput = 119.835'MB/sec zip pipeline throughput = 119.835'MB/sec entries = 200552 avg-key = 24.00 avg-zkey = 1.70 avg-val = 645.24 avg-zval = 646.30 usrkeys = 200552 avg-key = 16.00 avg-zkey = 1.70 avg-val = 645.82 avg-zval = 646.30 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 16694 bytes = 56352 } seq expand size = 116858 multi value expand size = 0 cnt WriteAppend = 15807 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1300 GB } UnZip/Zip{ index = 9.4369 value = 0.9984 dict = 0.00 all = 1.0321 } Zip/UnZip{ index = 0.1060 value = 1.0016 dict = 0.00 all = 0.9689 } ---------------------------- total value len = 0.911654 GB avg = 0.650 KB (by entry num) total key len = 0.033667 GB avg = 0.024 KB total ukey len = 0.022444 GB avg = 0.016 KB total ukey num = 0.001402779 Billion total entry num = 0.001402779 Billion write speed all = 0.024633472 MB/sec (with seq num) write speed all = 0.024341040 MB/sec (without seq num) 2024/11/20-19:27:17.481481 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61785: 200552 keys, 130048159 bytes, temperature: kWarm 2024/11/20-19:27:17.481546 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102037481516, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61785, "file_size": 130048159, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019979599, "table_properties": {"data_size": 129617360, "index_size": 340032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4813248, "raw_average_key_size": 24, "raw_value_size": 129404571, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200552, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102034, "oldest_key_time": 0, "file_creation_time": 1732102034, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61785, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:27:17.535985 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061786.sst) /dev/shm/ToplingTemp/Topling-886089-dtPDK5 2024/11/20-19:27:18.785801 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061786.sst): first pass time = 1.25's, 107.395'MB/sec 2024/11/20-19:27:20.016871 140008778016320 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061786.sst): part-0: reject ZipStore ratio = 0.800 2024/11/20-19:27:20.284545 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061786.sst): old prealloc_size = 147639500, real_size = 130014432 2024/11/20-19:27:20.284711 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061786.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200312 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 646.05 avg-zval = 647.11 2024/11/20-19:27:20.284859 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061786.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:27:20.591171 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061786.sst): second pass time = 1.23's, 105.190'MB/sec, value only(96.4% of KV) wait indexing time = 0.27's, remap KeyValue time = 0.31's, 425.284'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2324.141'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.203'MB/sec write SST data time = 0.06's, 2203.671'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.23's, unzip length = 0.129'GB zip my value throughput = 105.190'MB/sec zip pipeline throughput = 114.086'MB/sec entries = 200312 avg-key = 24.00 avg-zkey = 1.70 avg-val = 646.05 avg-zval = 647.11 usrkeys = 200312 avg-key = 16.00 avg-zkey = 1.70 avg-val = 650.18 avg-zval = 647.11 TagRS{ kinds = RS_Key0_TagN bytes = 31400 } TagArray{ size = 118247 bytes = 399104 } seq expand size = 827729 multi value expand size = 0 cnt WriteAppend = 15809 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1304 GB } UnZip/Zip{ index = 9.3973 value = 0.9984 dict = 0.00 all = 1.0293 } Zip/UnZip{ index = 0.1064 value = 1.0016 dict = 0.00 all = 0.9715 } ---------------------------- total value len = 1.041892 GB avg = 0.650 KB (by entry num) total key len = 0.038474 GB avg = 0.024 KB total ukey len = 0.025649 GB avg = 0.016 KB total ukey num = 0.001603091 Billion total entry num = 0.001603091 Billion write speed all = 0.028150113 MB/sec (with seq num) write speed all = 0.027815951 MB/sec (without seq num) 2024/11/20-19:27:20.791708 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61786: 200312 keys, 130397855 bytes, temperature: kWarm 2024/11/20-19:27:20.791776 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102040791744, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61786, "file_size": 130397855, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019989796, "table_properties": {"data_size": 129623120, "index_size": 341056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4807488, "raw_average_key_size": 24, "raw_value_size": 129410624, "raw_average_value_size": 646, "num_data_blocks": 1, "num_entries": 200312, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102037, "oldest_key_time": 0, "file_creation_time": 1732102037, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61786, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:27:20.860178 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061787.sst) /dev/shm/ToplingTemp/Topling-886089-vQIVU4 2024/11/20-19:27:22.223154 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061787.sst): first pass time = 1.36's, 98.478'MB/sec 2024/11/20-19:27:23.386042 140008778016320 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061787.sst): part-0: reject ZipStore ratio = 0.800 2024/11/20-19:27:23.641937 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061787.sst): old prealloc_size = 147639500, real_size = 130010592 2024/11/20-19:27:23.642108 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061787.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200493 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.44 avg-zval = 646.50 2024/11/20-19:27:23.642259 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061787.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:27:23.941346 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061787.sst): second pass time = 1.16's, 111.352'MB/sec, value only(96.4% of KV) wait indexing time = 0.26's, remap KeyValue time = 0.30's, 435.606'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2236.938'MB/sec (index lex order gen) rebuild zvType time = 0.24's, 0.209'MB/sec write SST data time = 0.06's, 2208.002'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.16's, unzip length = 0.129'GB zip my value throughput = 111.352'MB/sec zip pipeline throughput = 111.352'MB/sec entries = 200493 avg-key = 24.00 avg-zkey = 1.70 avg-val = 645.44 avg-zval = 646.50 usrkeys = 200493 avg-key = 16.00 avg-zkey = 1.70 avg-val = 649.96 avg-zval = 646.50 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 129442 bytes = 420704 } seq expand size = 906094 multi value expand size = 0 cnt WriteAppend = 15807 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1304 GB } UnZip/Zip{ index = 9.3987 value = 0.9984 dict = 0.00 all = 1.0292 } Zip/UnZip{ index = 0.1064 value = 1.0016 dict = 0.00 all = 0.9717 } ---------------------------- total value len = 1.172204 GB avg = 0.650 KB (by entry num) total key len = 0.043286 GB avg = 0.024 KB total ukey len = 0.028857 GB avg = 0.016 KB total ukey num = 0.001803584 Billion total entry num = 0.001803584 Billion write speed all = 0.031668156 MB/sec (with seq num) write speed all = 0.031292234 MB/sec (without seq num) 2024/11/20-19:27:24.131924 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61787: 200493 keys, 130415391 bytes, temperature: kWarm 2024/11/20-19:27:24.131993 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102044131961, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61787, "file_size": 130415391, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019984969, "table_properties": {"data_size": 129618960, "index_size": 341312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4811832, "raw_average_key_size": 24, "raw_value_size": 129406252, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200493, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102040, "oldest_key_time": 0, "file_creation_time": 1732102040, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61787, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:27:24.185208 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061788.sst) /dev/shm/ToplingTemp/Topling-886089-Vu6fL4 2024/11/20-19:27:25.750373 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061788.sst): first pass time = 1.57's, 85.755'MB/sec 2024/11/20-19:27:26.880992 140008778016320 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061788.sst): part-0: reject ZipStore ratio = 0.800 2024/11/20-19:27:27.151342 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061788.sst): old prealloc_size = 147639500, real_size = 130008896 2024/11/20-19:27:27.151510 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061788.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200554 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.24 avg-zval = 646.30 2024/11/20-19:27:27.151622 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061788.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:27:27.467387 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061788.sst): second pass time = 1.13's, 114.543'MB/sec, value only(96.4% of KV) wait indexing time = 0.27's, remap KeyValue time = 0.32's, 412.038'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2261.947'MB/sec (index lex order gen) rebuild zvType time = 0.26's, 0.194'MB/sec write SST data time = 0.06's, 2276.793'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.13's, unzip length = 0.129'GB zip my value throughput = 114.543'MB/sec zip pipeline throughput = 114.543'MB/sec entries = 200554 avg-key = 24.00 avg-zkey = 1.70 avg-val = 645.24 avg-zval = 646.30 usrkeys = 200554 avg-key = 16.00 avg-zkey = 1.70 avg-val = 647.63 avg-zval = 646.30 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 68718 bytes = 223344 } seq expand size = 481026 multi value expand size = 0 cnt WriteAppend = 15807 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1302 GB } UnZip/Zip{ index = 9.4016 value = 0.9984 dict = 0.00 all = 1.0307 } Zip/UnZip{ index = 0.1064 value = 1.0016 dict = 0.00 all = 0.9702 } ---------------------------- total value len = 1.302090 GB avg = 0.650 KB (by entry num) total key len = 0.048099 GB avg = 0.024 KB total ukey len = 0.032066 GB avg = 0.016 KB total ukey num = 0.002004138 Billion total entry num = 0.002004138 Billion write speed all = 0.035174341 MB/sec (with seq num) write speed all = 0.034756656 MB/sec (without seq num) 2024/11/20-19:27:27.635506 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61788: 200554 keys, 130216351 bytes, temperature: kWarm 2024/11/20-19:27:27.635564 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102047635533, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61788, "file_size": 130216351, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019989145, "table_properties": {"data_size": 129617264, "index_size": 341312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4813296, "raw_average_key_size": 24, "raw_value_size": 129404476, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200554, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102044, "oldest_key_time": 0, "file_creation_time": 1732102044, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61788, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:27:27.694674 140008778016320 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061789.sst) /dev/shm/ToplingTemp/Topling-886089-lLRGg6 2024/11/20-19:27:27.932220 140008778016320 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061789.sst): first pass time = 0.24's, 111.125'MB/sec 2024/11/20-19:27:28.250178 140008778016320 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061789.sst): part-0: reject ZipStore ratio = 0.808 2024/11/20-19:27:28.302630 140008778016320 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061789.sst): old prealloc_size = 147639500, real_size = 25566640 2024/11/20-19:27:28.302703 140008778016320 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061789.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 39423 prefix = 4 raw-key = 0.0005 GB zip-key = 0.0001 GB avg-key = 12.00 avg-zkey = 1.73 raw-val = 0.0254 GB zip-val = 0.0255 GB avg-val = 645.48 avg-zval = 646.54 2024/11/20-19:27:28.302768 140008778016320 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061789.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0001 GB, waited 0.000 sec, Key+Value = 0.026 GB 2024/11/20-19:27:28.365636 140008778016320 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00/061789.sst): second pass time = 0.32's, 80.204'MB/sec, value only(96.4% of KV) wait indexing time = 0.05's, remap KeyValue time = 0.06's, 407.504'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1197.515'MB/sec (index lex order gen) rebuild zvType time = 0.05's, 0.192'MB/sec write SST data time = 0.01's, 2212.050'MB/sec dict compress time = 2314885530.82's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.32's, unzip length = 0.025'GB zip my value throughput = 80.204'MB/sec zip pipeline throughput = 80.204'MB/sec entries = 39423 avg-key = 24.00 avg-zkey = 1.73 avg-val = 645.48 avg-zval = 646.54 usrkeys = 39423 avg-key = 16.00 avg-zkey = 1.73 avg-val = 651.03 avg-zval = 646.54 TagRS{ kinds = RS_Key0_TagN bytes = 6240 } TagArray{ size = 31246 bytes = 101568 } seq expand size = 218722 multi value expand size = 0 cnt WriteAppend = 3117 UnZipSize{ index = 0.0006 GB value = 0.0254 GB dict = 0.00 MB all = 0.0264 GB } __ZipSize{ index = 0.0001 GB value = 0.0255 GB dict = 0.00 MB all = 0.0257 GB } UnZip/Zip{ index = 9.2716 value = 0.9984 dict = 0.00 all = 1.0283 } Zip/UnZip{ index = 0.1079 value = 1.0016 dict = 0.00 all = 0.9725 } ---------------------------- total value len = 1.327755 GB avg = 0.650 KB (by entry num) total key len = 0.049045 GB avg = 0.024 KB total ukey len = 0.032697 GB avg = 0.016 KB total ukey num = 0.002043561 Billion total entry num = 0.002043561 Billion write speed all = 0.035866771 MB/sec (with seq num) write speed all = 0.035440880 MB/sec (without seq num) 2024/11/20-19:27:28.401992 140008778016320 [compaction_job.cc:1948] [default] [JOB 8083] Generated table #61789: 39423 keys, 25666078 bytes, temperature: kWarm 2024/11/20-19:27:28.402068 140008778016320 EVENT_LOG_v1 {"time_micros": 1732102048402023, "cf_name": "default", "job": 8083, "event": "table_file_creation", "file_number": 61789, "file_size": 25666078, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1019989823, "table_properties": {"data_size": 25488544, "index_size": 68032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 946152, "raw_average_key_size": 24, "raw_value_size": 25446698, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 39423, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102047, "oldest_key_time": 0, "file_creation_time": 1732102047, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61789, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:27:28.420244 140008778016320 [compaction_job.cc:703] job-08083: subcompact[0], size: 1.279966 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08083/att-00 [061779,061780,061781,061782,061783,061784,061785,061786,061787,061788,061789] 2024/11/20-19:27:28.549497 140008778016320 (Original Log Time 2024/11/20-19:26:54.349390) EVENT_LOG_v1 {"time_micros": 1732102014349350, "job": 8083, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [61773], "files_L3": [61706, 61707, 61708, 61709, 61710, 61711, 61712, 61713, 61714, 61717, 61718], "score": 0.976927, "input_data_size": 1275405377, "oldest_snapshot_seqno": 1020021358} 2024/11/20-19:27:28.549500 140008778016320 (Original Log Time 2024/11/20-19:27:28.548427) [compaction_job.cc:2022] [default] [JOB 8083] Compacted 1@2 + 11@3 files to L3 => 1279966100 bytes 2024/11/20-19:27:28.549502 140008778016320 (Original Log Time 2024/11/20-19:27:28.549418) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.05, MB/sec: 37.4 rd, 37.6 wr, level 3, files in(1, 11) out(11 +0 blob) MB in(21.8, 1194.6 +0.0 blob) out(1220.7 +0.0 blob), read-write-amplify(112.0) write-amplify(56.1) OK, records in: 2074109, records dropped: 30548 output_compression: Snappy 2024/11/20-19:27:28.549506 140008778016320 (Original Log Time 2024/11/20-19:27:28.549467) EVENT_LOG_v1 {"time_micros": 1732102048549443, "cf": "default", "job": 8083, "event": "compaction_finished", "compaction_time_micros": 34070861, "compaction_time_cpu_micros": 7507738, "output_level": 3, "num_output_files": 11, "total_output_size": 1279966100, "num_input_records": 2074109, "num_output_records": 2043561, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 831811662, "file_range_sync_nanos": 0, "file_fsync_nanos": 9928, "file_prepare_write_nanos": 4550084, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/20-19:27:28.684537 140008778016320 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732102014,"executesMs":34359,"compactionJobId":8083,"attempt":0,"compactionInputRawBytes":1389244311,"compactionInputZipBytes":1275405377,"compactionOutputRawBytes":0,"compactionOutputZipBytes":1279966100,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-9fdf32e0da775bfa15f5bcba89891377-10e64753be22c4d8-00"}}, response = 2024/11/20-19:27:28.684595 140008778016320 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-08083/att-00: olev 3, work 34.360 s, result 7.676 ms, install 1.094 ms, input{raw 1.294 GiB zip 1.188 GiB}