2024/11/20-19:28:57.690205 140008492664384 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00: fork to child time = 0.009489 sec 2024/11/20-19:28:57.690761 140008492664384 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00 2024/11/20-19:28:57.697348 140008492664384 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00 2024/11/20-19:28:57.699066 140008492664384 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-19:28:57.699298 140008492664384 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-19:28:57.760405 140008492664384 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-08157/att-00/MANIFEST-049091 2024/11/20-19:28:57.819818 140008492664384 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-08157/att-00/MANIFEST-049091 succeeded,manifest_file_number is 49091, next_file_number is 61917, last_sequence is 1021616121, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 61749 2024/11/20-19:28:57.819837 140008492664384 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-19:28:57.819962 140008492664384 [version_set.cc:5724] Creating manifest 61917 2024/11/20-19:28:57.852254 140008492664384 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-19:28:57.877024 140008492664384 [compaction_job.cc:2437] [default] [JOB 8157] Compacting 1@2 + 10@3 files to L3, score 0.83, subcompactions 1 : 1 2024/11/20-19:28:57.877059 140008492664384 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 7314 Base level 2, inputs: [61905(21MB)], [61804(124MB) 61805(124MB) 61806(124MB) 61807(124MB) 61808(100MB) 61809(100MB) 61810(124MB) 61811(124MB) 61812(124MB) 61914(124MB)] 2024/11/20-19:28:57.885157 140008492664384 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-19:28:57.885193 140008492664384 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-19:28:57.885221 140008492664384 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-19:28:57.885233 140008492664384 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-19:28:57.885248 140008492664384 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-19:28:59.279463 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061918.sst) /dev/shm/ToplingTemp/Topling-888030-4AcMXQ 2024/11/20-19:28:59.519675 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061918.sst): first pass time = 0.24's, 558.914'MB/sec 2024/11/20-19:29:00.471112 140008492664384 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061918.sst): part-0: reject ZipStore ratio = 0.803 2024/11/20-19:29:00.742199 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061918.sst): old prealloc_size = 147639500, real_size = 130030112 2024/11/20-19:29:00.742357 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061918.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200224 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0004 GB avg-key = 12.00 avg-zkey = 1.77 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 646.34 avg-zval = 647.40 2024/11/20-19:29:00.742588 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061918.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:29:01.050291 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061918.sst): second pass time = 0.95's, 136.145'MB/sec, value only(96.4% of KV) wait indexing time = 0.27's, remap KeyValue time = 0.31's, 424.180'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2572.634'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.199'MB/sec write SST data time = 0.06's, 2317.384'MB/sec dict compress time = 3548316928.70's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.95's, unzip length = 0.129'GB zip my value throughput = 136.145'MB/sec zip pipeline throughput = 136.145'MB/sec entries = 200224 avg-key = 24.00 avg-zkey = 1.77 avg-val = 646.34 avg-zval = 647.40 usrkeys = 200224 avg-key = 16.00 avg-zkey = 1.77 avg-val = 653.05 avg-zval = 647.40 TagRS{ kinds = RS_Key0_TagN bytes = 31400 } TagArray{ size = 192039 bytes = 672144 } seq expand size = 1344273 multi value expand size = 0 cnt WriteAppend = 15809 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0004 GB value = 0.1296 GB dict = 0.00 MB all = 0.1307 GB } UnZip/Zip{ index = 9.0240 value = 0.9984 dict = 0.00 all = 1.0270 } Zip/UnZip{ index = 0.1108 value = 1.0016 dict = 0.00 all = 0.9737 } ---------------------------- total value len = 0.130757 GB avg = 0.653 KB (by entry num) total key len = 0.004805 GB avg = 0.024 KB total ukey len = 0.003204 GB avg = 0.016 KB total ukey num = 0.000200224 Billion total entry num = 0.000200224 Billion write speed all = 0.003522995 MB/sec (with seq num) write speed all = 0.003481368 MB/sec (without seq num) 2024/11/20-19:29:01.218419 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61918: 200224 keys, 130686431 bytes, temperature: kWarm 2024/11/20-19:29:01.218527 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102141218473, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61918, "file_size": 130686431, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1021510574, "table_properties": {"data_size": 129624848, "index_size": 355008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4805376, "raw_average_key_size": 24, "raw_value_size": 129412423, "raw_average_value_size": 646, "num_data_blocks": 1, "num_entries": 200224, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102139, "oldest_key_time": 0, "file_creation_time": 1732102139, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61918, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:01.279676 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061919.sst) /dev/shm/ToplingTemp/Topling-888030-YbNkkR 2024/11/20-19:29:02.584460 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061919.sst): first pass time = 1.30's, 102.870'MB/sec 2024/11/20-19:29:03.769060 140008492664384 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061919.sst): part-0: reject ZipStore ratio = 0.802 2024/11/20-19:29:04.041227 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061919.sst): old prealloc_size = 147639500, real_size = 130022896 2024/11/20-19:29:04.041388 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061919.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200379 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0004 GB avg-key = 12.00 avg-zkey = 1.75 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.82 avg-zval = 646.88 2024/11/20-19:29:04.041570 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061919.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:29:04.351197 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061919.sst): second pass time = 1.18's, 109.303'MB/sec, value only(96.4% of KV) wait indexing time = 0.27's, remap KeyValue time = 0.31's, 421.102'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2437.194'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.200'MB/sec write SST data time = 0.06's, 2219.829'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.18's, unzip length = 0.129'GB zip my value throughput = 109.303'MB/sec zip pipeline throughput = 109.303'MB/sec entries = 200379 avg-key = 24.00 avg-zkey = 1.75 avg-val = 645.82 avg-zval = 646.88 usrkeys = 200379 avg-key = 16.00 avg-zkey = 1.75 avg-val = 650.99 avg-zval = 646.88 TagRS{ kinds = RS_Key0_TagN bytes = 31400 } TagArray{ size = 147945 bytes = 517824 } seq expand size = 1035615 multi value expand size = 0 cnt WriteAppend = 15809 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0004 GB value = 0.1296 GB dict = 0.00 MB all = 0.1305 GB } UnZip/Zip{ index = 9.1364 value = 0.9984 dict = 0.00 all = 1.0283 } Zip/UnZip{ index = 0.1095 value = 1.0016 dict = 0.00 all = 0.9725 } ---------------------------- total value len = 0.261201 GB avg = 0.652 KB (by entry num) total key len = 0.009614 GB avg = 0.024 KB total ukey len = 0.006410 GB avg = 0.016 KB total ukey num = 0.000400603 Billion total entry num = 0.000400603 Billion write speed all = 0.007037377 MB/sec (with seq num) write speed all = 0.006954097 MB/sec (without seq num) 2024/11/20-19:29:04.512813 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61919: 200379 keys, 130524831 bytes, temperature: kWarm 2024/11/20-19:29:04.512894 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102144512861, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61919, "file_size": 130524831, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1021508597, "table_properties": {"data_size": 129621728, "index_size": 350912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4809096, "raw_average_key_size": 24, "raw_value_size": 129409149, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200379, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102141, "oldest_key_time": 0, "file_creation_time": 1732102141, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61919, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:04.579831 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061920.sst) /dev/shm/ToplingTemp/Topling-888030-GpncyS 2024/11/20-19:29:06.035411 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061920.sst): first pass time = 1.46's, 92.212'MB/sec 2024/11/20-19:29:07.096837 140008492664384 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061920.sst): part-0: reject ZipStore ratio = 0.801 2024/11/20-19:29:07.437700 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061920.sst): old prealloc_size = 147639500, real_size = 130025840 2024/11/20-19:29:07.437894 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061920.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200003 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.73 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 647.08 avg-zval = 648.14 2024/11/20-19:29:07.438131 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061920.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:29:07.752961 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061920.sst): second pass time = 1.06's, 122.006'MB/sec, value only(96.4% of KV) wait indexing time = 0.34's, remap KeyValue time = 0.32's, 414.211'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2024.643'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.200'MB/sec write SST data time = 0.07's, 2001.453'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.06's, unzip length = 0.129'GB zip my value throughput = 122.006'MB/sec zip pipeline throughput = 122.006'MB/sec entries = 200003 avg-key = 24.00 avg-zkey = 1.73 avg-val = 647.08 avg-zval = 648.14 usrkeys = 200003 avg-key = 16.00 avg-zkey = 1.73 avg-val = 653.05 avg-zval = 648.14 TagRS{ kinds = RS_Key0_TagN bytes = 31360 } TagArray{ size = 170686 bytes = 576080 } seq expand size = 1194802 multi value expand size = 0 cnt WriteAppend = 15809 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1306 GB } UnZip/Zip{ index = 9.2560 value = 0.9984 dict = 0.00 all = 1.0278 } Zip/UnZip{ index = 0.1080 value = 1.0016 dict = 0.00 all = 0.9729 } ---------------------------- total value len = 0.391814 GB avg = 0.652 KB (by entry num) total key len = 0.014415 GB avg = 0.024 KB total ukey len = 0.009610 GB avg = 0.016 KB total ukey num = 0.000600606 Billion total entry num = 0.000600606 Billion write speed all = 0.010555251 MB/sec (with seq num) write speed all = 0.010430404 MB/sec (without seq num) 2024/11/20-19:29:07.918522 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61920: 200003 keys, 130586079 bytes, temperature: kWarm 2024/11/20-19:29:07.918587 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102147918556, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61920, "file_size": 130586079, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1021510404, "table_properties": {"data_size": 129629920, "index_size": 345728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4800072, "raw_average_key_size": 24, "raw_value_size": 129417665, "raw_average_value_size": 647, "num_data_blocks": 1, "num_entries": 200003, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102144, "oldest_key_time": 0, "file_creation_time": 1732102144, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61920, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:07.976980 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061921.sst) /dev/shm/ToplingTemp/Topling-888030-UjJWSS 2024/11/20-19:29:09.296467 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061921.sst): first pass time = 1.32's, 101.723'MB/sec 2024/11/20-19:29:10.465504 140008492664384 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061921.sst): part-0: reject ZipStore ratio = 0.801 2024/11/20-19:29:10.840118 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061921.sst): old prealloc_size = 147639500, real_size = 130014320 2024/11/20-19:29:10.840273 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061921.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200424 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.71 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.67 avg-zval = 646.73 2024/11/20-19:29:10.840409 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061921.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:29:11.163636 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061921.sst): second pass time = 1.17's, 110.769'MB/sec, value only(96.4% of KV) wait indexing time = 0.37's, remap KeyValue time = 0.32's, 402.946'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2481.820'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.199'MB/sec write SST data time = 0.07's, 1806.416'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.17's, unzip length = 0.129'GB zip my value throughput = 110.769'MB/sec zip pipeline throughput = 110.769'MB/sec entries = 200424 avg-key = 24.00 avg-zkey = 1.71 avg-val = 645.67 avg-zval = 646.73 usrkeys = 200424 avg-key = 16.00 avg-zkey = 1.71 avg-val = 649.35 avg-zval = 646.73 TagRS{ kinds = RS_Key0_TagN bytes = 31400 } TagArray{ size = 105424 bytes = 355824 } seq expand size = 737968 multi value expand size = 0 cnt WriteAppend = 15808 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1304 GB } UnZip/Zip{ index = 9.3307 value = 0.9984 dict = 0.00 all = 1.0296 } Zip/UnZip{ index = 0.1072 value = 1.0016 dict = 0.00 all = 0.9712 } ---------------------------- total value len = 0.521960 GB avg = 0.652 KB (by entry num) total key len = 0.019225 GB avg = 0.024 KB total ukey len = 0.012816 GB avg = 0.016 KB total ukey num = 0.000801030 Billion total entry num = 0.000801030 Billion write speed all = 0.014060633 MB/sec (with seq num) write speed all = 0.013894139 MB/sec (without seq num) 2024/11/20-19:29:11.321893 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61921: 200424 keys, 130354207 bytes, temperature: kWarm 2024/11/20-19:29:11.321954 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102151321924, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61921, "file_size": 130354207, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1021509066, "table_properties": {"data_size": 129620320, "index_size": 343680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4810176, "raw_average_key_size": 24, "raw_value_size": 129407667, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102147, "oldest_key_time": 0, "file_creation_time": 1732102147, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61921, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:11.377614 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061922.sst) /dev/shm/ToplingTemp/Topling-888030-2q0FGQ 2024/11/20-19:29:12.466985 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061922.sst): first pass time = 1.09's, 123.211'MB/sec 2024/11/20-19:29:13.456608 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061922.sst): old prealloc_size = 147639500, real_size = 103893264 2024/11/20-19:29:13.456785 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061922.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200526 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1035 GB avg-val = 645.33 avg-zval = 516.16 2024/11/20-19:29:13.456945 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061922.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:29:13.827482 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061922.sst): second pass time = 0.99's, 130.873'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 285.054'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2604.420'MB/sec (index lex order gen) rebuild zvType time = 0.29's, 0.171'MB/sec write SST data time = 0.08's, 1377.611'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.349'MB, throughput = 23951.688'MB/sec zip my value time = 0.99's, unzip length = 0.129'GB zip my value throughput = 130.873'MB/sec zip pipeline throughput = 130.873'MB/sec entries = 200526 avg-key = 24.00 avg-zkey = 1.69 avg-val = 645.33 avg-zval = 516.16 usrkeys = 200526 avg-key = 16.00 avg-zkey = 1.69 avg-val = 650.37 avg-zval = 516.16 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 144371 bytes = 487264 } seq expand size = 1010597 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.35 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1035 GB dict = 1.35 MB all = 0.1057 GB } UnZip/Zip{ index = 9.4623 value = 1.2502 dict = 1.00 all = 1.2696 } Zip/UnZip{ index = 0.1057 value = 0.7998 dict = 1.00 all = 0.7876 } ---------------------------- total value len = 0.652376 GB avg = 0.651 KB (by entry num) total key len = 0.024037 GB avg = 0.024 KB total ukey len = 0.016025 GB avg = 0.016 KB total ukey num = 0.001001556 Billion total entry num = 0.001001556 Billion write speed all = 0.017572830 MB/sec (with seq num) write speed all = 0.017364671 MB/sec (without seq num) 2024/11/20-19:29:13.962350 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61922: 200526 keys, 105713119 bytes, temperature: kWarm 2024/11/20-19:29:13.962418 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102153962386, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61922, "file_size": 105713119, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1021508031, "table_properties": {"data_size": 103503872, "index_size": 339072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4812624, "raw_average_key_size": 24, "raw_value_size": 129405680, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200526, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102151, "oldest_key_time": 0, "file_creation_time": 1732102151, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61922, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:14.014510 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061923.sst) /dev/shm/ToplingTemp/Topling-888030-DxmF6S 2024/11/20-19:29:15.374058 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061923.sst): first pass time = 1.36's, 98.727'MB/sec 2024/11/20-19:29:16.704899 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061923.sst): old prealloc_size = 147639500, real_size = 103871568 2024/11/20-19:29:16.705078 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061923.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200639 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1035 GB avg-val = 644.95 avg-zval = 515.76 2024/11/20-19:29:16.705182 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061923.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:29:16.986652 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061923.sst): second pass time = 1.33's, 97.292'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.28's, 374.048'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2151.175'MB/sec (index lex order gen) rebuild zvType time = 0.22's, 0.229'MB/sec write SST data time = 0.06's, 1687.120'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.348'MB, throughput = 35906.440'MB/sec zip my value time = 1.33's, unzip length = 0.129'GB zip my value throughput = 97.292'MB/sec zip pipeline throughput = 156.577'MB/sec entries = 200639 avg-key = 24.00 avg-zkey = 1.69 avg-val = 644.95 avg-zval = 515.76 usrkeys = 200639 avg-key = 16.00 avg-zkey = 1.69 avg-val = 646.84 avg-zval = 515.76 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 53937 bytes = 182048 } seq expand size = 377559 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.35 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1035 GB dict = 1.35 MB all = 0.1054 GB } UnZip/Zip{ index = 9.4481 value = 1.2505 dict = 1.00 all = 1.2736 } Zip/UnZip{ index = 0.1058 value = 0.7997 dict = 1.00 all = 0.7852 } ---------------------------- total value len = 0.782157 GB avg = 0.651 KB (by entry num) total key len = 0.028853 GB avg = 0.024 KB total ukey len = 0.019235 GB avg = 0.016 KB total ukey num = 0.001202195 Billion total entry num = 0.001202195 Billion write speed all = 0.021067829 MB/sec (with seq num) write speed all = 0.020817990 MB/sec (without seq num) 2024/11/20-19:29:17.123979 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61923: 200639 keys, 105385918 bytes, temperature: kWarm 2024/11/20-19:29:17.124058 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102157124015, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61923, "file_size": 105385918, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1021501578, "table_properties": {"data_size": 103481472, "index_size": 339776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4815336, "raw_average_key_size": 24, "raw_value_size": 129403103, "raw_average_value_size": 644, "num_data_blocks": 1, "num_entries": 200639, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102154, "oldest_key_time": 0, "file_creation_time": 1732102154, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61923, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:17.171767 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061924.sst) /dev/shm/ToplingTemp/Topling-888030-3brcyS 2024/11/20-19:29:18.458344 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061924.sst): first pass time = 1.29's, 104.325'MB/sec 2024/11/20-19:29:19.853764 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061924.sst): old prealloc_size = 147639500, real_size = 103908624 2024/11/20-19:29:19.853981 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061924.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200555 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.1294 GB zip-val = 0.1035 GB avg-val = 645.23 avg-zval = 516.16 2024/11/20-19:29:19.854093 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061924.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:29:20.226682 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061924.sst): second pass time = 1.39's, 92.798'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 282.348'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1786.117'MB/sec (index lex order gen) rebuild zvType time = 0.31's, 0.163'MB/sec write SST data time = 0.06's, 1640.197'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.351'MB, throughput = 32127.667'MB/sec zip my value time = 1.39's, unzip length = 0.129'GB zip my value throughput = 92.798'MB/sec zip pipeline throughput = 201.287'MB/sec entries = 200555 avg-key = 24.00 avg-zkey = 1.70 avg-val = 645.23 avg-zval = 516.16 usrkeys = 200555 avg-key = 16.00 avg-zkey = 1.70 avg-val = 645.71 avg-zval = 516.16 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 13670 bytes = 46144 } seq expand size = 95690 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.35 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1035 GB dict = 1.35 MB all = 0.1053 GB } UnZip/Zip{ index = 9.4370 value = 1.2501 dict = 1.00 all = 1.2748 } Zip/UnZip{ index = 0.1060 value = 0.8000 dict = 1.00 all = 0.7845 } ---------------------------- total value len = 0.911657 GB avg = 0.650 KB (by entry num) total key len = 0.033666 GB avg = 0.024 KB total ukey len = 0.022444 GB avg = 0.016 KB total ukey num = 0.001402750 Billion total entry num = 0.001402750 Billion write speed all = 0.024554877 MB/sec (with seq num) write speed all = 0.024263385 MB/sec (without seq num) 2024/11/20-19:29:20.364584 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61924: 200555 keys, 105289502 bytes, temperature: kWarm 2024/11/20-19:29:20.364664 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102160364624, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61924, "file_size": 105289502, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1021502918, "table_properties": {"data_size": 103518272, "index_size": 340032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4813320, "raw_average_key_size": 24, "raw_value_size": 129405011, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200555, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102157, "oldest_key_time": 0, "file_creation_time": 1732102157, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61924, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:20.412031 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061925.sst) /dev/shm/ToplingTemp/Topling-888030-MYfTLS 2024/11/20-19:29:21.858525 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061925.sst): first pass time = 1.45's, 92.792'MB/sec 2024/11/20-19:29:23.136807 140008492664384 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061925.sst): part-0: reject ZipStore ratio = 0.800 2024/11/20-19:29:23.442301 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061925.sst): old prealloc_size = 147639500, real_size = 130014704 2024/11/20-19:29:23.442498 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061925.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200305 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 646.07 avg-zval = 647.13 2024/11/20-19:29:23.442680 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061925.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:29:23.840511 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061925.sst): second pass time = 1.28's, 101.301'MB/sec, value only(96.4% of KV) wait indexing time = 0.31's, remap KeyValue time = 0.40's, 327.465'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1950.452'MB/sec (index lex order gen) rebuild zvType time = 0.34's, 0.149'MB/sec write SST data time = 0.06's, 2076.931'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.28's, unzip length = 0.129'GB zip my value throughput = 101.301'MB/sec zip pipeline throughput = 164.438'MB/sec entries = 200305 avg-key = 24.00 avg-zkey = 1.70 avg-val = 646.07 avg-zval = 647.13 usrkeys = 200305 avg-key = 16.00 avg-zkey = 1.70 avg-val = 650.20 avg-zval = 647.13 TagRS{ kinds = RS_Key0_TagN bytes = 31400 } TagArray{ size = 118282 bytes = 399216 } seq expand size = 827974 multi value expand size = 0 cnt WriteAppend = 15809 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1304 GB } UnZip/Zip{ index = 9.3969 value = 0.9984 dict = 0.00 all = 1.0293 } Zip/UnZip{ index = 0.1064 value = 1.0016 dict = 0.00 all = 0.9715 } ---------------------------- total value len = 1.041896 GB avg = 0.650 KB (by entry num) total key len = 0.038473 GB avg = 0.024 KB total ukey len = 0.025649 GB avg = 0.016 KB total ukey num = 0.001603055 Billion total entry num = 0.001603055 Billion write speed all = 0.028060083 MB/sec (with seq num) write speed all = 0.027726998 MB/sec (without seq num) 2024/11/20-19:29:24.002438 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61925: 200305 keys, 130398239 bytes, temperature: kWarm 2024/11/20-19:29:24.002516 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102164002482, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61925, "file_size": 130398239, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1021510635, "table_properties": {"data_size": 129623392, "index_size": 341056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4807320, "raw_average_key_size": 24, "raw_value_size": 129410873, "raw_average_value_size": 646, "num_data_blocks": 1, "num_entries": 200305, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102160, "oldest_key_time": 0, "file_creation_time": 1732102160, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61925, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:24.070247 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061926.sst) /dev/shm/ToplingTemp/Topling-888030-g0RJHP 2024/11/20-19:29:25.548624 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061926.sst): first pass time = 1.48's, 90.791'MB/sec 2024/11/20-19:29:26.702921 140008492664384 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061926.sst): part-0: reject ZipStore ratio = 0.800 2024/11/20-19:29:27.003449 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061926.sst): old prealloc_size = 147639500, real_size = 130010832 2024/11/20-19:29:27.003658 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061926.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 200491 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.1294 GB zip-val = 0.1296 GB avg-val = 645.45 avg-zval = 646.51 2024/11/20-19:29:27.003843 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061926.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:29:27.468388 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061926.sst): second pass time = 1.15's, 112.165'MB/sec, value only(96.4% of KV) wait indexing time = 0.30's, remap KeyValue time = 0.46's, 280.517'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1945.893'MB/sec (index lex order gen) rebuild zvType time = 0.38's, 0.131'MB/sec write SST data time = 0.08's, 1565.581'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 1.15's, unzip length = 0.129'GB zip my value throughput = 112.165'MB/sec zip pipeline throughput = 112.165'MB/sec entries = 200491 avg-key = 24.00 avg-zkey = 1.70 avg-val = 645.45 avg-zval = 646.51 usrkeys = 200491 avg-key = 16.00 avg-zkey = 1.70 avg-val = 649.97 avg-zval = 646.51 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 129469 bytes = 420784 } seq expand size = 906283 multi value expand size = 0 cnt WriteAppend = 15806 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 0.00 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1296 GB dict = 0.00 MB all = 0.1304 GB } UnZip/Zip{ index = 9.3951 value = 0.9984 dict = 0.00 all = 1.0292 } Zip/UnZip{ index = 0.1064 value = 1.0016 dict = 0.00 all = 0.9717 } ---------------------------- total value len = 1.172209 GB avg = 0.650 KB (by entry num) total key len = 0.043285 GB avg = 0.024 KB total ukey len = 0.028857 GB avg = 0.016 KB total ukey num = 0.001803546 Billion total entry num = 0.001803546 Billion write speed all = 0.031566652 MB/sec (with seq num) write speed all = 0.031191944 MB/sec (without seq num) 2024/11/20-19:29:27.670164 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61926: 200491 keys, 130415711 bytes, temperature: kWarm 2024/11/20-19:29:27.670233 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102167670190, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61926, "file_size": 130415711, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1021510246, "table_properties": {"data_size": 129619072, "index_size": 341440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4811784, "raw_average_key_size": 24, "raw_value_size": 129406367, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200491, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102164, "oldest_key_time": 0, "file_creation_time": 1732102164, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61926, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:27.763401 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061927.sst) /dev/shm/ToplingTemp/Topling-888030-hFE56P 2024/11/20-19:29:29.238970 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061927.sst): first pass time = 1.48's, 90.963'MB/sec 2024/11/20-19:29:30.559140 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061927.sst): old prealloc_size = 147639500, real_size = 103900880 2024/11/20-19:29:30.559317 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061927.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 200573 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.1294 GB zip-val = 0.1035 GB avg-val = 645.17 avg-zval = 516.07 2024/11/20-19:29:30.559435 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061927.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-19:29:30.927926 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061927.sst): second pass time = 1.32's, 98.089'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.37's, 285.984'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2393.989'MB/sec (index lex order gen) rebuild zvType time = 0.27's, 0.183'MB/sec write SST data time = 0.09's, 1120.613'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.350'MB, throughput = 46530.675'MB/sec zip my value time = 1.32's, unzip length = 0.129'GB zip my value throughput = 98.089'MB/sec zip pipeline throughput = 124.044'MB/sec entries = 200573 avg-key = 24.00 avg-zkey = 1.69 avg-val = 645.17 avg-zval = 516.07 usrkeys = 200573 avg-key = 16.00 avg-zkey = 1.69 avg-val = 647.58 avg-zval = 516.07 TagRS{ kinds = RS_Key0_TagN bytes = 31440 } TagArray{ size = 68809 bytes = 223648 } seq expand size = 481663 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0032 GB value = 0.1294 GB dict = 1.35 MB all = 0.1342 GB } __ZipSize{ index = 0.0003 GB value = 0.1035 GB dict = 1.35 MB all = 0.1055 GB } UnZip/Zip{ index = 9.4396 value = 1.2502 dict = 1.00 all = 1.2727 } Zip/UnZip{ index = 0.1059 value = 0.7999 dict = 1.00 all = 0.7857 } ---------------------------- total value len = 1.302095 GB avg = 0.650 KB (by entry num) total key len = 0.048099 GB avg = 0.024 KB total ukey len = 0.032066 GB avg = 0.016 KB total ukey num = 0.002004119 Billion total entry num = 0.002004119 Billion write speed all = 0.035061692 MB/sec (with seq num) write speed all = 0.034645350 MB/sec (without seq num) 2024/11/20-19:29:31.067590 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61927: 200573 keys, 105458656 bytes, temperature: kWarm 2024/11/20-19:29:31.067661 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102171067628, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61927, "file_size": 105458656, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 1021508392, "table_properties": {"data_size": 103510592, "index_size": 339968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4813752, "raw_average_key_size": 24, "raw_value_size": 129404608, "raw_average_value_size": 645, "num_data_blocks": 1, "num_entries": 200573, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732102167, "oldest_key_time": 0, "file_creation_time": 1732102167, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61927, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:31.120733 140008492664384 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061928.sst) /dev/shm/ToplingTemp/Topling-888030-ZPDhwR 2024/11/20-19:29:31.135803 140008492664384 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061928.sst): first pass time = 0.02's, 179.323'MB/sec 2024/11/20-19:29:31.188213 140008492664384 [top_zip_table_builder.cc:2752] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061928.sst): part-0: reject ZipStore ratio = 0.833 2024/11/20-19:29:31.193547 140008492664384 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061928.sst): old prealloc_size = 147639500, real_size = 2615376 2024/11/20-19:29:31.193588 140008492664384 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061928.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 4008 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 12.00 avg-zkey = 2.46 raw-val = 0.0026 GB zip-val = 0.0026 GB avg-val = 648.67 avg-zval = 649.79 2024/11/20-19:29:31.193618 140008492664384 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061928.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0000 GB, waited 0.000 sec, Key+Value = 0.003 GB 2024/11/20-19:29:31.201710 140008492664384 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/061928.sst): second pass time = 0.05's, 50.185'MB/sec, value only(96.4% of KV) wait indexing time = 0.01's, remap KeyValue time = 0.01's, 322.824'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 291.546'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.146'MB/sec write SST data time = 0.00's, 2099.083'MB/sec dict compress time = 140029.08's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.05's, unzip length = 0.003'GB zip my value throughput = 50.185'MB/sec zip pipeline throughput = 50.185'MB/sec entries = 4008 avg-key = 24.00 avg-zkey = 2.46 avg-val = 648.67 avg-zval = 649.79 usrkeys = 4008 avg-key = 16.00 avg-zkey = 2.46 avg-val = 655.67 avg-zval = 649.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 4008 bytes = 13040 } seq expand size = 28056 multi value expand size = 0 cnt WriteAppend = 325 UnZipSize{ index = 0.0001 GB value = 0.0026 GB dict = 0.00 MB all = 0.0027 GB } __ZipSize{ index = 0.0000 GB value = 0.0026 GB dict = 0.00 MB all = 0.0026 GB } UnZip/Zip{ index = 6.5065 value = 0.9983 dict = 0.00 all = 1.0257 } Zip/UnZip{ index = 0.1537 value = 1.0017 dict = 0.00 all = 0.9750 } ---------------------------- total value len = 1.304723 GB avg = 0.650 KB (by entry num) total key len = 0.048195 GB avg = 0.024 KB total ukey len = 0.032130 GB avg = 0.016 KB total ukey num = 0.002008127 Billion total entry num = 0.002008127 Billion write speed all = 0.035132182 MB/sec (with seq num) write speed all = 0.034715010 MB/sec (without seq num) 2024/11/20-19:29:31.214105 140008492664384 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #61928: 4008 keys, 2628571 bytes, temperature: kWarm 2024/11/20-19:29:31.214154 140008492664384 EVENT_LOG_v1 {"time_micros": 1732102171214127, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 61928, "file_size": 2628571, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 979142221, "largest_seqno": 1021493513, "table_properties": {"data_size": 2604352, "index_size": 9856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 96192, "raw_average_key_size": 24, "raw_value_size": 2599871, "raw_average_value_size": 648, "num_data_blocks": 1, "num_entries": 4008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732102171, "oldest_key_time": 0, "file_creation_time": 1732102171, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "6MZ3RU376XPAYLWHL32R", "orig_file_number": 61928, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-19:29:31.224232 140008492664384 [compaction_job.cc:703] job-08157: subcompact[0], size: 1.207441 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00 [061918,061919,061920,061921,061922,061923,061924,061925,061926,061927,061928] 2024/11/20-19:29:31.354122 140008492664384 (Original Log Time 2024/11/20-19:28:57.877143) EVENT_LOG_v1 {"time_micros": 1732102137877082, "job": 8157, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [61905], "files_L3": [61804, 61805, 61806, 61807, 61808, 61809, 61810, 61811, 61812, 61914], "score": 0.829697, "input_data_size": 1277333968, "oldest_snapshot_seqno": 1021616081} 2024/11/20-19:29:31.354125 140008492664384 (Original Log Time 2024/11/20-19:29:31.353091) [compaction_job.cc:2022] [default] [JOB 8157] Compacted 1@2 + 10@3 files to L3 => 1207441264 bytes 2024/11/20-19:29:31.354126 140008492664384 (Original Log Time 2024/11/20-19:29:31.354032) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.04, MB/sec: 38.3 rd, 36.2 wr, level 3, files in(1, 10) out(11 +0 blob) MB in(22.0, 1196.2 +0.0 blob) out(1151.5 +0.0 blob), read-write-amplify(107.9) write-amplify(52.4) OK, records in: 2039008, records dropped: 30881 output_compression: Snappy 2024/11/20-19:29:31.354130 140008492664384 (Original Log Time 2024/11/20-19:29:31.354100) EVENT_LOG_v1 {"time_micros": 1732102171354072, "cf": "default", "job": 8157, "event": "compaction_finished", "compaction_time_micros": 33347092, "compaction_time_cpu_micros": 7533870, "output_level": 3, "num_output_files": 11, "total_output_size": 1207441264, "num_input_records": 2039008, "num_output_records": 2008127, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 886507738, "file_range_sync_nanos": 0, "file_fsync_nanos": 9284, "file_prepare_write_nanos": 3866695, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/20-19:29:31.485188 140008492664384 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732096383,"starts":1732102137,"executesMs":33655,"compactionJobId":8157,"attempt":0,"compactionInputRawBytes":1365769948,"compactionInputZipBytes":1277333968,"compactionOutputRawBytes":0,"compactionOutputZipBytes":1207441264,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-d93fb2a12e1d6fd1f49704794379bf3a-c55b84890f1ffc3f-00"}}, response = 2024/11/20-19:29:31.485267 140008492664384 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T17.53.03/.rocksdb/job-08157/att-00: olev 3, work 33.655 s, result 7.650 ms, install 1.055 ms, input{raw 1.272 GiB zip 1.190 GiB}