2024/11/20-13:23:57.627169 140257928148544 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00: fork to child time = 0.009796 sec 2024/11/20-13:23:57.627673 140257928148544 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00 2024/11/20-13:23:57.635605 140257928148544 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00 2024/11/20-13:23:57.637198 140257928148544 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-13:23:57.637421 140257928148544 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-13:23:57.685954 140257928148544 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-07223/att-00/MANIFEST-019309 2024/11/20-13:23:57.741630 140257928148544 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-07223/att-00/MANIFEST-019309 succeeded,manifest_file_number is 19309, next_file_number is 30002, last_sequence is 568113890, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 29821 2024/11/20-13:23:57.741641 140257928148544 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-13:23:57.741742 140257928148544 [version_set.cc:5724] Creating manifest 30002 2024/11/20-13:23:57.798565 140257928148544 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-13:23:57.819590 140257928148544 [compaction_job.cc:2437] [default] [JOB 7223] Compacting 1@2 + 23@3 files to L3, score 0.79, subcompactions 1 : 1 2024/11/20-13:23:57.819610 140257928148544 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 6645 Base level 2, inputs: [29981(13MB)], [29998(40MB) 29999(505KB) 29929(2020KB) 29881(39MB) 29882(39MB) 29912(39MB) 29911(39MB) 29913(39MB) 29914(39MB) 29916(39MB) 29915(39MB) 29917(39MB) 29918(39MB) 29919(39MB) 29920(39MB) 29921(39MB) 29922(39MB) 29923(39MB) 29930(39MB) 29931(39MB) 29941(39MB) 29943(39MB) 29942(30MB)] 2024/11/20-13:23:57.824374 140257928148544 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-13:23:57.824398 140257928148544 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-13:23:57.824415 140257928148544 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-13:23:57.824422 140257928148544 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-13:23:57.824433 140257928148544 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-13:23:58.353108 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030003.sst) /dev/shm/ToplingTemp/Topling-232298-nXKhXv 2024/11/20-13:23:58.791928 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030003.sst): first pass time = 0.44's, 305.906'MB/sec 2024/11/20-13:23:59.474894 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030003.sst): old prealloc_size = 147639500, real_size = 40622416 2024/11/20-13:23:59.475016 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030003.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.01 avg-zval = 311.91 2024/11/20-13:23:59.475141 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030003.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:23:59.636313 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030003.sst): second pass time = 0.68's, 192.938'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.098'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2256.791'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 961.640'MB/sec dict compress time = 7596570295.46's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 27011.448'MB/sec zip my value time = 0.68's, unzip length = 0.132'GB zip my value throughput = 192.938'MB/sec zip pipeline throughput = 363.434'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.01 avg-zval = 311.91 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1019.79 avg-zval = 311.91 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 51355 bytes = 160496 } seq expand size = 359485 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 7.0628 value = 3.2606 dict = 1.00 all = 3.1843 } Zip/UnZip{ index = 0.1416 value = 0.3067 dict = 1.00 all = 0.3140 } ---------------------------- total value len = 0.131990 GB avg = 1.020 KB (by entry num) total key len = 0.002589 GB avg = 0.020 KB total ukey len = 0.001553 GB avg = 0.012 KB total ukey num = 0.000129428 Billion total entry num = 0.000129428 Billion write speed all = 0.008117982 MB/sec (with seq num) write speed all = 0.008055523 MB/sec (without seq num) 2024/11/20-13:23:59.809899 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30003: 129428 keys, 42150542 bytes, temperature: kWarm 2024/11/20-13:23:59.809998 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080239809955, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30003, "file_size": 42150542, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567939848, "table_properties": {"data_size": 40369984, "index_size": 219904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131630211, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080238, "oldest_key_time": 0, "file_creation_time": 1732080238, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30003, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:23:59.843577 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030004.sst) /dev/shm/ToplingTemp/Topling-232298-4v3ZWt 2024/11/20-13:24:00.457388 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030004.sst): first pass time = 0.61's, 218.680'MB/sec 2024/11/20-13:24:01.139987 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030004.sst): old prealloc_size = 147639500, real_size = 40504448 2024/11/20-13:24:01.140131 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030004.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.03 avg-zval = 310.98 2024/11/20-13:24:01.140199 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030004.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:01.302315 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030004.sst): second pass time = 0.68's, 193.019'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.477'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1814.554'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 936.210'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 19466.341'MB/sec zip my value time = 0.68's, unzip length = 0.132'GB zip my value throughput = 193.019'MB/sec zip pipeline throughput = 363.204'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.03 avg-zval = 310.98 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.43 avg-zval = 310.98 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 25900 bytes = 80960 } seq expand size = 181300 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9654 value = 3.2704 dict = 1.00 all = 3.1992 } Zip/UnZip{ index = 0.1436 value = 0.3058 dict = 1.00 all = 0.3126 } ---------------------------- total value len = 0.263801 GB avg = 1.019 KB (by entry num) total key len = 0.005177 GB avg = 0.020 KB total ukey len = 0.003106 GB avg = 0.012 KB total ukey num = 0.000258854 Billion total entry num = 0.000258854 Billion write speed all = 0.016223577 MB/sec (with seq num) write speed all = 0.016098673 MB/sec (without seq num) 2024/11/20-13:24:01.387381 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30004: 129426 keys, 41953409 bytes, temperature: kWarm 2024/11/20-13:24:01.387449 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080241387417, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30004, "file_size": 41953409, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567939850, "table_properties": {"data_size": 40248944, "index_size": 222976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131630123, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080239, "oldest_key_time": 0, "file_creation_time": 1732080239, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30004, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:01.422417 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030005.sst) /dev/shm/ToplingTemp/Topling-232298-DK6VKu 2024/11/20-13:24:01.984509 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030005.sst): first pass time = 0.56's, 238.796'MB/sec 2024/11/20-13:24:03.128842 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030005.sst): old prealloc_size = 147639500, real_size = 40358016 2024/11/20-13:24:03.128975 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030005.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.86 2024/11/20-13:24:03.129028 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030005.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:03.284841 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030005.sst): second pass time = 1.14's, 115.113'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.537'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2014.606'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.284'MB/sec write SST data time = 0.04's, 995.306'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 38635.747'MB/sec zip my value time = 1.14's, unzip length = 0.132'GB zip my value throughput = 115.113'MB/sec zip pipeline throughput = 295.888'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1016.98 avg-zval = 309.86 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1017.07 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 1527 bytes = 4784 } seq expand size = 10689 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0465 value = 3.2821 dict = 1.00 all = 3.2163 } Zip/UnZip{ index = 0.1419 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.395441 GB avg = 1.018 KB (by entry num) total key len = 0.007766 GB avg = 0.020 KB total ukey len = 0.004659 GB avg = 0.012 KB total ukey num = 0.000388285 Billion total entry num = 0.000388285 Billion write speed all = 0.024316740 MB/sec (with seq num) write speed all = 0.024129405 MB/sec (without seq num) 2024/11/20-13:24:03.380003 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30005: 129431 keys, 41730764 bytes, temperature: kWarm 2024/11/20-13:24:03.380093 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080243380041, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30005, "file_size": 41730764, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567912293, "table_properties": {"data_size": 40105072, "index_size": 220416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629207, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080241, "oldest_key_time": 0, "file_creation_time": 1732080241, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30005, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:03.417891 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030006.sst) /dev/shm/ToplingTemp/Topling-232298-0kXX3s 2024/11/20-13:24:03.993393 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030006.sst): first pass time = 0.58's, 233.233'MB/sec 2024/11/20-13:24:04.676963 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030006.sst): old prealloc_size = 147639500, real_size = 40354016 2024/11/20-13:24:04.677091 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030006.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.85 2024/11/20-13:24:04.677142 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030006.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:04.833377 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030006.sst): second pass time = 0.68's, 192.767'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.775'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1999.634'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.284'MB/sec write SST data time = 0.04's, 987.456'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 20760.319'MB/sec zip my value time = 0.68's, unzip length = 0.132'GB zip my value throughput = 192.767'MB/sec zip pipeline throughput = 306.072'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 19 bytes = 64 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.527071 GB avg = 1.018 KB (by entry num) total key len = 0.010354 GB avg = 0.020 KB total ukey len = 0.006213 GB avg = 0.012 KB total ukey num = 0.000517713 Billion total entry num = 0.000517713 Billion write speed all = 0.032408203 MB/sec (with seq num) write speed all = 0.032158447 MB/sec (without seq num) 2024/11/20-13:24:04.884389 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30006: 129428 keys, 41721896 bytes, temperature: kWarm 2024/11/20-13:24:04.884451 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080244884422, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30006, "file_size": 41721896, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567741082, "table_properties": {"data_size": 40102672, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629657, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080243, "oldest_key_time": 0, "file_creation_time": 1732080243, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30006, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:04.919757 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030007.sst) /dev/shm/ToplingTemp/Topling-232298-XRY3Wr 2024/11/20-13:24:05.454726 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030007.sst): first pass time = 0.53's, 250.905'MB/sec 2024/11/20-13:24:06.645037 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030007.sst): old prealloc_size = 147639500, real_size = 40356480 2024/11/20-13:24:06.645171 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030007.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.04 avg-zval = 309.87 2024/11/20-13:24:06.645221 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030007.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:06.802405 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030007.sst): second pass time = 1.19's, 110.657'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.154'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2335.083'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.282'MB/sec write SST data time = 0.04's, 981.033'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 30692.601'MB/sec zip my value time = 1.19's, unzip length = 0.132'GB zip my value throughput = 110.657'MB/sec zip pipeline throughput = 309.757'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.04 avg-zval = 309.87 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.04 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 16 bytes = 64 } seq expand size = 112 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0977 value = 3.2821 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.658701 GB avg = 1.018 KB (by entry num) total key len = 0.012943 GB avg = 0.020 KB total ukey len = 0.007766 GB avg = 0.012 KB total ukey num = 0.000647138 Billion total entry num = 0.000647138 Billion write speed all = 0.040497131 MB/sec (with seq num) write speed all = 0.040184974 MB/sec (without seq num) 2024/11/20-13:24:06.869637 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30007: 129425 keys, 41724613 bytes, temperature: kWarm 2024/11/20-13:24:06.869694 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080246869664, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30007, "file_size": 41724613, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567877298, "table_properties": {"data_size": 40105136, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629775, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080244, "oldest_key_time": 0, "file_creation_time": 1732080244, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30007, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:06.912219 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030008.sst) /dev/shm/ToplingTemp/Topling-232298-ivf6ds 2024/11/20-13:24:07.486016 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030008.sst): first pass time = 0.57's, 233.927'MB/sec 2024/11/20-13:24:08.093594 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030008.sst): old prealloc_size = 147639500, real_size = 40357376 2024/11/20-13:24:08.093727 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030008.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.87 2024/11/20-13:24:08.093782 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030008.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:08.280548 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030008.sst): second pass time = 0.61's, 217.003'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 223.205'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2107.628'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.224'MB/sec write SST data time = 0.04's, 989.411'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 24910.983'MB/sec zip my value time = 0.61's, unzip length = 0.132'GB zip my value throughput = 217.003'MB/sec zip pipeline throughput = 229.069'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.87 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 12 bytes = 48 } seq expand size = 84 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2820 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.790330 GB avg = 1.018 KB (by entry num) total key len = 0.015531 GB avg = 0.020 KB total ukey len = 0.009319 GB avg = 0.012 KB total ukey num = 0.000776565 Billion total entry num = 0.000776565 Billion write speed all = 0.048585566 MB/sec (with seq num) write speed all = 0.048211011 MB/sec (without seq num) 2024/11/20-13:24:08.330509 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30008: 129427 keys, 41725086 bytes, temperature: kWarm 2024/11/20-13:24:08.330572 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080248330542, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30008, "file_size": 41725086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567927132, "table_properties": {"data_size": 40106032, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629589, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080246, "oldest_key_time": 0, "file_creation_time": 1732080246, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30008, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:08.364228 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030009.sst) /dev/shm/ToplingTemp/Topling-232298-CPwDus 2024/11/20-13:24:08.942934 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030009.sst): first pass time = 0.58's, 231.941'MB/sec 2024/11/20-13:24:09.505842 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030009.sst): old prealloc_size = 147639500, real_size = 40352624 2024/11/20-13:24:09.505998 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030009.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.83 2024/11/20-13:24:09.506091 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030009.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:09.826663 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030009.sst): second pass time = 0.56's, 234.185'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 130.044'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1854.986'MB/sec (index lex order gen) rebuild zvType time = 0.24's, 0.136'MB/sec write SST data time = 0.08's, 504.163'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 27826.573'MB/sec zip my value time = 0.56's, unzip length = 0.132'GB zip my value throughput = 234.185'MB/sec zip pipeline throughput = 234.185'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.83 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.83 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 24 bytes = 80 } seq expand size = 168 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2824 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 0.921960 GB avg = 1.018 KB (by entry num) total key len = 0.018120 GB avg = 0.020 KB total ukey len = 0.010872 GB avg = 0.012 KB total ukey num = 0.000905997 Billion total entry num = 0.000905997 Billion write speed all = 0.056672351 MB/sec (with seq num) write speed all = 0.056235410 MB/sec (without seq num) 2024/11/20-13:24:09.875790 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30009: 129432 keys, 41720037 bytes, temperature: kWarm 2024/11/20-13:24:09.875855 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080249875825, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30009, "file_size": 41720037, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567924505, "table_properties": {"data_size": 40101280, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629815, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080248, "oldest_key_time": 0, "file_creation_time": 1732080248, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30009, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:09.909981 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030010.sst) /dev/shm/ToplingTemp/Topling-232298-Clse5t 2024/11/20-13:24:10.589601 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030010.sst): first pass time = 0.68's, 197.499'MB/sec 2024/11/20-13:24:11.059124 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030010.sst): old prealloc_size = 147639500, real_size = 40350288 2024/11/20-13:24:11.059237 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030010.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.97 avg-zval = 309.80 2024/11/20-13:24:11.059288 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030010.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:11.215078 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030010.sst): second pass time = 0.47's, 280.816'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.562'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2402.565'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.280'MB/sec write SST data time = 0.04's, 1040.300'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40421.422'MB/sec zip my value time = 0.47's, unzip length = 0.132'GB zip my value throughput = 280.816'MB/sec zip pipeline throughput = 280.816'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.80 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.80 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0982 value = 3.2826 dict = 1.00 all = 3.2173 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.053590 GB avg = 1.018 KB (by entry num) total key len = 0.020709 GB avg = 0.020 KB total ukey len = 0.012425 GB avg = 0.012 KB total ukey num = 0.001035430 Billion total entry num = 0.001035430 Billion write speed all = 0.064758231 MB/sec (with seq num) write speed all = 0.064258909 MB/sec (without seq num) 2024/11/20-13:24:11.265287 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30010: 129433 keys, 41717876 bytes, temperature: kWarm 2024/11/20-13:24:11.265341 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080251265313, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30010, "file_size": 41717876, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567921176, "table_properties": {"data_size": 40098944, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629607, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080249, "oldest_key_time": 0, "file_creation_time": 1732080249, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30010, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:11.300568 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030011.sst) /dev/shm/ToplingTemp/Topling-232298-UrZOAt 2024/11/20-13:24:11.830562 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030011.sst): first pass time = 0.53's, 253.261'MB/sec 2024/11/20-13:24:12.384612 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030011.sst): old prealloc_size = 147639500, real_size = 40355744 2024/11/20-13:24:12.384740 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030011.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.86 2024/11/20-13:24:12.384812 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030011.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:12.769539 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030011.sst): second pass time = 0.55's, 237.920'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.38's, 108.406'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1960.893'MB/sec (index lex order gen) rebuild zvType time = 0.27's, 0.120'MB/sec write SST data time = 0.12's, 361.948'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46218.120'MB/sec zip my value time = 0.55's, unzip length = 0.132'GB zip my value throughput = 237.920'MB/sec zip pipeline throughput = 245.943'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.86 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 16 bytes = 64 } seq expand size = 112 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2822 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.185220 GB avg = 1.017 KB (by entry num) total key len = 0.023297 GB avg = 0.020 KB total ukey len = 0.013978 GB avg = 0.012 KB total ukey num = 0.001164856 Billion total entry num = 0.001164856 Billion write speed all = 0.072842040 MB/sec (with seq num) write speed all = 0.072280357 MB/sec (without seq num) 2024/11/20-13:24:12.821243 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30011: 129426 keys, 41723834 bytes, temperature: kWarm 2024/11/20-13:24:12.821307 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080252821277, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30011, "file_size": 41723834, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567912566, "table_properties": {"data_size": 40104400, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629957, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080251, "oldest_key_time": 0, "file_creation_time": 1732080251, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30011, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:12.856933 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030012.sst) /dev/shm/ToplingTemp/Topling-232298-WK3h8t 2024/11/20-13:24:13.386249 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030012.sst): first pass time = 0.53's, 253.586'MB/sec 2024/11/20-13:24:13.907731 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030012.sst): old prealloc_size = 147639500, real_size = 40354544 2024/11/20-13:24:13.907847 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030012.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.85 2024/11/20-13:24:13.907909 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030012.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:14.084713 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030012.sst): second pass time = 0.52's, 252.747'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 235.807'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2360.322'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.06's, 689.034'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44214.792'MB/sec zip my value time = 0.52's, unzip length = 0.132'GB zip my value throughput = 252.747'MB/sec zip pipeline throughput = 252.747'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 21 bytes = 64 } seq expand size = 147 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.316850 GB avg = 1.017 KB (by entry num) total key len = 0.025886 GB avg = 0.020 KB total ukey len = 0.015531 GB avg = 0.012 KB total ukey num = 0.001294283 Billion total entry num = 0.001294283 Billion write speed all = 0.080925490 MB/sec (with seq num) write speed all = 0.080301447 MB/sec (without seq num) 2024/11/20-13:24:14.136827 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30012: 129427 keys, 41722273 bytes, temperature: kWarm 2024/11/20-13:24:14.136925 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080254136872, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30012, "file_size": 41722273, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567925895, "table_properties": {"data_size": 40103200, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629693, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080252, "oldest_key_time": 0, "file_creation_time": 1732080252, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30012, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:14.181897 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030013.sst) /dev/shm/ToplingTemp/Topling-232298-5P7yJs 2024/11/20-13:24:14.701805 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030013.sst): first pass time = 0.52's, 258.181'MB/sec 2024/11/20-13:24:15.232744 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030013.sst): old prealloc_size = 147639500, real_size = 40354128 2024/11/20-13:24:15.232915 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030013.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.85 2024/11/20-13:24:15.232991 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030013.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:15.591863 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030013.sst): second pass time = 0.53's, 248.324'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.36's, 116.191'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1510.586'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.140'MB/sec write SST data time = 0.13's, 326.515'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44302.893'MB/sec zip my value time = 0.53's, unzip length = 0.132'GB zip my value throughput = 248.324'MB/sec zip pipeline throughput = 248.324'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 19 bytes = 64 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.448479 GB avg = 1.017 KB (by entry num) total key len = 0.028474 GB avg = 0.020 KB total ukey len = 0.017085 GB avg = 0.012 KB total ukey num = 0.001423710 Billion total entry num = 0.001423710 Billion write speed all = 0.089006607 MB/sec (with seq num) write speed all = 0.088320223 MB/sec (without seq num) 2024/11/20-13:24:15.646610 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30013: 129427 keys, 41722134 bytes, temperature: kWarm 2024/11/20-13:24:15.646699 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080255646654, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30013, "file_size": 41722134, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567937153, "table_properties": {"data_size": 40102784, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629359, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080254, "oldest_key_time": 0, "file_creation_time": 1732080254, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30013, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:15.683865 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030014.sst) /dev/shm/ToplingTemp/Topling-232298-fR4j2r 2024/11/20-13:24:16.223551 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030014.sst): first pass time = 0.54's, 248.725'MB/sec 2024/11/20-13:24:16.726628 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030014.sst): old prealloc_size = 147639500, real_size = 40350256 2024/11/20-13:24:16.726773 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030014.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.81 2024/11/20-13:24:16.726834 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030014.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:16.883458 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030014.sst): second pass time = 0.50's, 262.504'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.010'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1973.378'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.282'MB/sec write SST data time = 0.04's, 990.300'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 25397.284'MB/sec zip my value time = 0.50's, unzip length = 0.132'GB zip my value throughput = 262.504'MB/sec zip pipeline throughput = 262.504'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.81 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.81 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 11 bytes = 48 } seq expand size = 77 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2826 dict = 1.00 all = 3.2172 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.580109 GB avg = 1.017 KB (by entry num) total key len = 0.031063 GB avg = 0.020 KB total ukey len = 0.018638 GB avg = 0.012 KB total ukey num = 0.001553141 Billion total entry num = 0.001553141 Billion write speed all = 0.097087522 MB/sec (with seq num) write speed all = 0.096338797 MB/sec (without seq num) 2024/11/20-13:24:16.936089 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30014: 129431 keys, 41718494 bytes, temperature: kWarm 2024/11/20-13:24:16.936151 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080256936122, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30014, "file_size": 41718494, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567933650, "table_properties": {"data_size": 40098912, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629413, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080255, "oldest_key_time": 0, "file_creation_time": 1732080255, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30014, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:16.969039 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030015.sst) /dev/shm/ToplingTemp/Topling-232298-m3y97t 2024/11/20-13:24:17.510494 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030015.sst): first pass time = 0.54's, 247.910'MB/sec 2024/11/20-13:24:18.215961 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030015.sst): old prealloc_size = 147639500, real_size = 40354640 2024/11/20-13:24:18.216147 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030015.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.85 2024/11/20-13:24:18.216223 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030015.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:18.519342 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030015.sst): second pass time = 0.69's, 189.995'MB/sec, value only(98.1% of KV) wait indexing time = 0.01's, remap KeyValue time = 0.30's, 137.524'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1527.394'MB/sec (index lex order gen) rebuild zvType time = 0.23's, 0.143'MB/sec write SST data time = 0.08's, 544.209'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45306.390'MB/sec zip my value time = 0.69's, unzip length = 0.132'GB zip my value throughput = 189.995'MB/sec zip pipeline throughput = 189.995'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 19 bytes = 64 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.711739 GB avg = 1.017 KB (by entry num) total key len = 0.033651 GB avg = 0.020 KB total ukey len = 0.020191 GB avg = 0.012 KB total ukey num = 0.001682570 Billion total entry num = 0.001682570 Billion write speed all = 0.105165034 MB/sec (with seq num) write speed all = 0.104353995 MB/sec (without seq num) 2024/11/20-13:24:18.568557 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30015: 129429 keys, 41722105 bytes, temperature: kWarm 2024/11/20-13:24:18.568620 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080258568590, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30015, "file_size": 41722105, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567822507, "table_properties": {"data_size": 40103296, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630006, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080256, "oldest_key_time": 0, "file_creation_time": 1732080256, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30015, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:18.601444 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030016.sst) /dev/shm/ToplingTemp/Topling-232298-XQqA6t 2024/11/20-13:24:19.076995 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030016.sst): first pass time = 0.48's, 282.257'MB/sec 2024/11/20-13:24:19.554036 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030016.sst): old prealloc_size = 147639500, real_size = 40356976 2024/11/20-13:24:19.554159 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030016.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.87 2024/11/20-13:24:19.554212 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030016.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:19.708769 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030016.sst): second pass time = 0.48's, 276.397'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 269.691'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2375.852'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.282'MB/sec write SST data time = 0.04's, 1050.154'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 24358.439'MB/sec zip my value time = 0.48's, unzip length = 0.132'GB zip my value throughput = 276.397'MB/sec zip pipeline throughput = 276.397'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.87 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2821 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.843369 GB avg = 1.017 KB (by entry num) total key len = 0.036240 GB avg = 0.020 KB total ukey len = 0.021744 GB avg = 0.012 KB total ukey num = 0.001811996 Billion total entry num = 0.001811996 Billion write speed all = 0.113243977 MB/sec (with seq num) write speed all = 0.112370613 MB/sec (without seq num) 2024/11/20-13:24:19.759699 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30016: 129426 keys, 41725122 bytes, temperature: kWarm 2024/11/20-13:24:19.759760 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080259759731, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30016, "file_size": 41725122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567929700, "table_properties": {"data_size": 40105632, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629675, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080258, "oldest_key_time": 0, "file_creation_time": 1732080258, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30016, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:19.795177 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030017.sst) /dev/shm/ToplingTemp/Topling-232298-99u2yt 2024/11/20-13:24:20.319458 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030017.sst): first pass time = 0.52's, 256.022'MB/sec 2024/11/20-13:24:20.819648 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030017.sst): old prealloc_size = 147639500, real_size = 40354272 2024/11/20-13:24:20.819754 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030017.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.85 2024/11/20-13:24:20.819801 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030017.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:21.001567 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030017.sst): second pass time = 0.50's, 263.566'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 229.400'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2542.510'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.262'MB/sec write SST data time = 0.06's, 718.236'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 25293.852'MB/sec zip my value time = 0.50's, unzip length = 0.132'GB zip my value throughput = 263.566'MB/sec zip pipeline throughput = 263.566'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.85 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 19 bytes = 64 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.974999 GB avg = 1.017 KB (by entry num) total key len = 0.038828 GB avg = 0.020 KB total ukey len = 0.023297 GB avg = 0.012 KB total ukey num = 0.001941422 Billion total entry num = 0.001941422 Billion write speed all = 0.121321015 MB/sec (with seq num) write speed all = 0.120385342 MB/sec (without seq num) 2024/11/20-13:24:21.056036 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30017: 129426 keys, 41721831 bytes, temperature: kWarm 2024/11/20-13:24:21.056135 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080261056090, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30017, "file_size": 41721831, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567771459, "table_properties": {"data_size": 40102928, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629816, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080259, "oldest_key_time": 0, "file_creation_time": 1732080259, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30017, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:21.097683 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030018.sst) /dev/shm/ToplingTemp/Topling-232298-fGB1fv 2024/11/20-13:24:21.815165 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030018.sst): first pass time = 0.72's, 187.079'MB/sec 2024/11/20-13:24:22.315257 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030018.sst): old prealloc_size = 147639500, real_size = 40352592 2024/11/20-13:24:22.315377 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030018.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.83 2024/11/20-13:24:22.315431 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030018.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:22.479154 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030018.sst): second pass time = 0.50's, 263.580'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 254.580'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2321.581'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.05's, 882.124'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 27353.608'MB/sec zip my value time = 0.50's, unzip length = 0.132'GB zip my value throughput = 263.580'MB/sec zip pipeline throughput = 263.580'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.83 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.83 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 12 bytes = 48 } seq expand size = 84 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2824 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 2.106629 GB avg = 1.017 KB (by entry num) total key len = 0.041417 GB avg = 0.020 KB total ukey len = 0.024850 GB avg = 0.012 KB total ukey num = 0.002070852 Billion total entry num = 0.002070852 Billion write speed all = 0.129395352 MB/sec (with seq num) write speed all = 0.128397390 MB/sec (without seq num) 2024/11/20-13:24:22.609354 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30018: 129430 keys, 41720273 bytes, temperature: kWarm 2024/11/20-13:24:22.609416 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080262609387, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30018, "file_size": 41720273, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567862515, "table_properties": {"data_size": 40101248, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629758, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080261, "oldest_key_time": 0, "file_creation_time": 1732080261, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30018, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:22.657448 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030019.sst) /dev/shm/ToplingTemp/Topling-232298-titueu 2024/11/20-13:24:23.175099 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030019.sst): first pass time = 0.52's, 259.302'MB/sec 2024/11/20-13:24:23.658521 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030019.sst): old prealloc_size = 147639500, real_size = 40354192 2024/11/20-13:24:23.658633 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030019.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.85 2024/11/20-13:24:23.658685 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030019.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:23.814622 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030019.sst): second pass time = 0.48's, 272.729'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.309'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2428.213'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.282'MB/sec write SST data time = 0.04's, 1008.534'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 25593.519'MB/sec zip my value time = 0.48's, unzip length = 0.132'GB zip my value throughput = 272.729'MB/sec zip pipeline throughput = 272.729'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 12 bytes = 48 } seq expand size = 84 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.238258 GB avg = 1.017 KB (by entry num) total key len = 0.044006 GB avg = 0.020 KB total ukey len = 0.026403 GB avg = 0.012 KB total ukey num = 0.002200280 Billion total entry num = 0.002200280 Billion write speed all = 0.137469412 MB/sec (with seq num) write speed all = 0.136409162 MB/sec (without seq num) 2024/11/20-13:24:23.865147 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30019: 129428 keys, 41722146 bytes, temperature: kWarm 2024/11/20-13:24:23.865211 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080263865180, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30019, "file_size": 41722146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567907701, "table_properties": {"data_size": 40102848, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629499, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080262, "oldest_key_time": 0, "file_creation_time": 1732080262, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30019, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:23.903205 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030020.sst) /dev/shm/ToplingTemp/Topling-232298-iZFEXv 2024/11/20-13:24:24.716826 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030020.sst): first pass time = 0.81's, 164.971'MB/sec 2024/11/20-13:24:25.328079 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030020.sst): old prealloc_size = 147639500, real_size = 40352144 2024/11/20-13:24:25.328184 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030020.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.82 2024/11/20-13:24:25.328229 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030020.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:25.481193 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030020.sst): second pass time = 0.61's, 215.585'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 272.487'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2536.438'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.286'MB/sec write SST data time = 0.04's, 1047.441'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 57122.766'MB/sec zip my value time = 0.61's, unzip length = 0.132'GB zip my value throughput = 215.585'MB/sec zip pipeline throughput = 224.161'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.82 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.82 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 19 bytes = 64 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2825 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 2.369888 GB avg = 1.017 KB (by entry num) total key len = 0.046594 GB avg = 0.020 KB total ukey len = 0.027957 GB avg = 0.012 KB total ukey num = 0.002329711 Billion total entry num = 0.002329711 Billion write speed all = 0.145539255 MB/sec (with seq num) write speed all = 0.144416749 MB/sec (without seq num) 2024/11/20-13:24:25.529988 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30020: 129431 keys, 41719631 bytes, temperature: kWarm 2024/11/20-13:24:25.530055 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080265530017, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30020, "file_size": 41719631, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567859456, "table_properties": {"data_size": 40100800, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629154, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080263, "oldest_key_time": 0, "file_creation_time": 1732080263, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30020, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:25.563770 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030021.sst) /dev/shm/ToplingTemp/Topling-232298-KxsUgs 2024/11/20-13:24:26.168891 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030021.sst): first pass time = 0.61's, 221.816'MB/sec 2024/11/20-13:24:26.659718 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030021.sst): old prealloc_size = 147639500, real_size = 40354384 2024/11/20-13:24:26.659848 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030021.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.85 2024/11/20-13:24:26.659921 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030021.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:26.822289 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030021.sst): second pass time = 0.49's, 268.548'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 256.648'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2134.582'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.04's, 968.453'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 31469.137'MB/sec zip my value time = 0.49's, unzip length = 0.132'GB zip my value throughput = 268.548'MB/sec zip pipeline throughput = 268.548'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 16 bytes = 64 } seq expand size = 112 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.501518 GB avg = 1.017 KB (by entry num) total key len = 0.049183 GB avg = 0.020 KB total ukey len = 0.029510 GB avg = 0.012 KB total ukey num = 0.002459139 Billion total entry num = 0.002459139 Billion write speed all = 0.153610534 MB/sec (with seq num) write speed all = 0.152425762 MB/sec (without seq num) 2024/11/20-13:24:26.872024 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30021: 129428 keys, 41722192 bytes, temperature: kWarm 2024/11/20-13:24:26.872097 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080266872068, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30021, "file_size": 41722192, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567935061, "table_properties": {"data_size": 40103040, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629957, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080265, "oldest_key_time": 0, "file_creation_time": 1732080265, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30021, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:26.905863 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030022.sst) /dev/shm/ToplingTemp/Topling-232298-cyUStu 2024/11/20-13:24:27.584154 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030022.sst): first pass time = 0.68's, 197.892'MB/sec 2024/11/20-13:24:28.179539 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030022.sst): old prealloc_size = 147639500, real_size = 40356880 2024/11/20-13:24:28.179652 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030022.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129424 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.04 avg-zval = 309.88 2024/11/20-13:24:28.179706 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030022.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-13:24:28.333182 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030022.sst): second pass time = 0.59's, 221.348'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 271.591'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2326.939'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.286'MB/sec write SST data time = 0.04's, 1033.753'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46287.336'MB/sec zip my value time = 0.59's, unzip length = 0.132'GB zip my value throughput = 221.348'MB/sec zip pipeline throughput = 353.347'MB/sec entries = 129424 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.04 avg-zval = 309.88 usrkeys = 129424 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.04 avg-zval = 309.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0977 value = 3.2821 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.633148 GB avg = 1.017 KB (by entry num) total key len = 0.051771 GB avg = 0.020 KB total ukey len = 0.031063 GB avg = 0.012 KB total ukey num = 0.002588563 Billion total entry num = 0.002588563 Billion write speed all = 0.161678847 MB/sec (with seq num) write speed all = 0.160431835 MB/sec (without seq num) 2024/11/20-13:24:28.384926 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30022: 129424 keys, 41724921 bytes, temperature: kWarm 2024/11/20-13:24:28.384991 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080268384960, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30022, "file_size": 41724921, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567909018, "table_properties": {"data_size": 40105536, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588480, "raw_average_key_size": 20, "raw_value_size": 131629925, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080266, "oldest_key_time": 0, "file_creation_time": 1732080266, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30022, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:28.421102 140257928148544 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030023.sst) /dev/shm/ToplingTemp/Topling-232298-4oih1t 2024/11/20-13:24:28.976178 140257928148544 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030023.sst): first pass time = 0.56's, 203.187'MB/sec 2024/11/20-13:24:29.449116 140257928148544 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030023.sst): old prealloc_size = 147639500, real_size = 33934032 2024/11/20-13:24:29.449218 140257928148544 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030023.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 108757 prefix = 4 raw-key = 0.0009 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1106 GB zip-val = 0.0337 GB avg-val =1016.97 avg-zval = 310.07 2024/11/20-13:24:29.449267 140257928148544 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030023.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.113 GB 2024/11/20-13:24:29.579099 140257928148544 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00/030023.sst): second pass time = 0.47's, 234.180'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.13's, 269.926'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2173.320'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.282'MB/sec write SST data time = 0.03's, 1051.272'MB/sec dict compress time = -1697615388.21's, -0.000'MB/sec z-dict build time = 0.00's, sample length = 1.151'MB, throughput = 38032.705'MB/sec zip my value time = 0.47's, unzip length = 0.111'GB zip my value throughput = 234.180'MB/sec zip pipeline throughput = 234.180'MB/sec entries = 108757 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 310.07 usrkeys = 108757 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 310.07 TagRS{ kinds = RS_Key0_TagN bytes = 17080 } TagArray{ size = 12 bytes = 48 } seq expand size = 84 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0013 GB value = 0.1106 GB dict = 1.15 MB all = 0.1128 GB } __ZipSize{ index = 0.0002 GB value = 0.0337 GB dict = 1.15 MB all = 0.0351 GB } UnZip/Zip{ index = 7.0879 value = 3.2798 dict = 1.00 all = 3.2152 } Zip/UnZip{ index = 0.1411 value = 0.3049 dict = 1.00 all = 0.3110 } ---------------------------- total value len = 2.743751 GB avg = 1.017 KB (by entry num) total key len = 0.053946 GB avg = 0.020 KB total ukey len = 0.032368 GB avg = 0.012 KB total ukey num = 0.002697320 Billion total entry num = 0.002697320 Billion write speed all = 0.168457411 MB/sec (with seq num) write speed all = 0.167158104 MB/sec (without seq num) 2024/11/20-13:24:29.624219 140257928148544 [compaction_job.cc:1948] [default] [JOB 7223] Generated table #30023: 108757 keys, 35076855 bytes, temperature: kWarm 2024/11/20-13:24:29.624282 140257928148544 EVENT_LOG_v1 {"time_micros": 1732080269624252, "cf_name": "default", "job": 7223, "event": "table_file_creation", "file_number": 30023, "file_size": 35076855, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 567937742, "table_properties": {"data_size": 33722560, "index_size": 184128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2175140, "raw_average_key_size": 20, "raw_value_size": 110602842, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 108757, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732080268, "oldest_key_time": 0, "file_creation_time": 1732080268, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 30023, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-13:24:29.647666 140257928148544 [compaction_job.cc:703] job-07223: subcompact[0], size: 0.870186 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-07223/att-00 [030003,030004,030005,030006,030007,030008,030009,030010,030011,030012,030013,030014,030015,030016,030017,030018,030019,030020,030021,030022,030023] 2024/11/20-13:24:29.805636 140257928148544 (Original Log Time 2024/11/20-13:23:57.819669) EVENT_LOG_v1 {"time_micros": 1732080237819626, "job": 7223, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [29981], "files_L3": [29998, 29999, 29929, 29881, 29882, 29912, 29911, 29913, 29914, 29916, 29915, 29917, 29918, 29919, 29920, 29921, 29922, 29923, 29930, 29931, 29941, 29943, 29942], "score": 0.79189, "input_data_size": 883636216, "oldest_snapshot_seqno": 568113749} 2024/11/20-13:24:29.805638 140257928148544 (Original Log Time 2024/11/20-13:24:29.803935) [compaction_job.cc:2022] [default] [JOB 7223] Compacted 1@2 + 23@3 files to L3 => 870186034 bytes 2024/11/20-13:24:29.805640 140257928148544 (Original Log Time 2024/11/20-13:24:29.805563) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 27.8 rd, 27.3 wr, level 3, files in(1, 23) out(21 +0 blob) MB in(13.4, 829.3 +0.0 blob) out(829.9 +0.0 blob), read-write-amplify(125.1) write-amplify(62.1) OK, records in: 2710153, records dropped: 12833 output_compression: Snappy 2024/11/20-13:24:29.805643 140257928148544 (Original Log Time 2024/11/20-13:24:29.805611) EVENT_LOG_v1 {"time_micros": 1732080269805585, "cf": "default", "job": 7223, "event": "compaction_finished", "compaction_time_micros": 31828003, "compaction_time_cpu_micros": 7459855, "output_level": 3, "num_output_files": 21, "total_output_size": 870186034, "num_input_records": 2710153, "num_output_records": 2697320, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 756198999, "file_range_sync_nanos": 0, "file_fsync_nanos": 20340, "file_prepare_write_nanos": 45184, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/20-13:24:29.938158 140257928148544 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732076400,"starts":1732080237,"executesMs":32167,"compactionJobId":7223,"attempt":0,"compactionInputRawBytes":2810452304,"compactionInputZipBytes":883636216,"compactionOutputRawBytes":0,"compactionOutputZipBytes":870186034,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-40812584ea8b0b4ed3ce9bd296041243-abbb1b6d4e17d7bd-00"}}, response = 2024/11/20-13:24:29.938212 140257928148544 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-07223/att-00: olev 3, work 32.168 s, result 8.974 ms, install 1.722 ms, input{raw 2.617 GiB zip 842.701 MiB}