2024/11/20-14:05:01.898650 140257819043392 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00: fork to child time = 0.008225 sec 2024/11/20-14:05:01.899144 140257819043392 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00 2024/11/20-14:05:01.907768 140257819043392 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00 2024/11/20-14:05:01.909385 140257819043392 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-14:05:01.909600 140257819043392 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-14:05:01.956932 140257819043392 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-08911/att-00/MANIFEST-019309 2024/11/20-14:05:01.998816 140257819043392 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-08911/att-00/MANIFEST-019309 succeeded,manifest_file_number is 19309, next_file_number is 33197, last_sequence is 607255828, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 33057 2024/11/20-14:05:01.998823 140257819043392 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-14:05:01.998881 140257819043392 [version_set.cc:5724] Creating manifest 33197 2024/11/20-14:05:02.027833 140257819043392 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-14:05:02.046916 140257819043392 [compaction_job.cc:2437] [default] [JOB 8911] Compacting 1@2 + 7@3 files to L3, score 0.85, subcompactions 1 : 1 2024/11/20-14:05:02.046928 140257819043392 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 7906 Base level 2, inputs: [33194(31MB)], [33122(40MB) 33123(1609KB) 33110(40MB) 33111(39MB) 33112(39MB) 33113(39MB) 33125(40MB)] 2024/11/20-14:05:02.054014 140257819043392 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-14:05:02.054037 140257819043392 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-14:05:02.054062 140257819043392 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-14:05:02.054070 140257819043392 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-14:05:02.054081 140257819043392 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-14:05:02.656906 140257819043392 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033198.sst) /dev/shm/ToplingTemp/Topling-272819-rLCurw 2024/11/20-14:05:02.834553 140257819043392 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033198.sst): first pass time = 0.18's, 755.739'MB/sec 2024/11/20-14:05:03.452815 140257819043392 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033198.sst): old prealloc_size = 147639500, real_size = 40959936 2024/11/20-14:05:03.452943 140257819043392 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033198.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1016.98 avg-zval = 314.41 2024/11/20-14:05:03.453123 140257819043392 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033198.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-14:05:03.624554 140257819043392 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033198.sst): second pass time = 0.62's, 213.163'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 248.865'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2281.561'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.255'MB/sec write SST data time = 0.04's, 955.437'MB/sec dict compress time = 8247586391.95's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40105.401'MB/sec zip my value time = 0.62's, unzip length = 0.132'GB zip my value throughput = 213.163'MB/sec zip pipeline throughput = 277.645'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.98 avg-zval = 314.41 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.98 avg-zval = 314.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 436848 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.6507 value = 3.2346 dict = 1.00 all = 3.1400 } Zip/UnZip{ index = 0.1504 value = 0.3092 dict = 1.00 all = 0.3185 } ---------------------------- total value len = 0.132535 GB avg = 1.024 KB (by entry num) total key len = 0.002589 GB avg = 0.020 KB total ukey len = 0.001553 GB avg = 0.012 KB total ukey num = 0.000129431 Billion total entry num = 0.000129431 Billion write speed all = 0.007096172 MB/sec (with seq num) write speed all = 0.007041794 MB/sec (without seq num) 2024/11/20-14:05:03.676175 140257819043392 [compaction_job.cc:1948] [default] [JOB 8911] Generated table #33198: 129431 keys, 42744325 bytes, temperature: kWarm 2024/11/20-14:05:03.676256 140257819043392 EVENT_LOG_v1 {"time_micros": 1732082703676217, "cf_name": "default", "job": 8911, "event": "table_file_creation", "file_number": 33198, "file_size": 42744325, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 476056905, "largest_seqno": 607192616, "table_properties": {"data_size": 40693872, "index_size": 233536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629150, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732082702, "oldest_key_time": 0, "file_creation_time": 1732082702, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 33198, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-14:05:03.716479 140257819043392 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033199.sst) /dev/shm/ToplingTemp/Topling-272819-Klddbt 2024/11/20-14:05:04.257405 140257819043392 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033199.sst): first pass time = 0.54's, 248.142'MB/sec 2024/11/20-14:05:04.773015 140257819043392 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033199.sst): old prealloc_size = 147639500, real_size = 40661696 2024/11/20-14:05:04.773170 140257819043392 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033199.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.02 avg-zval = 312.22 2024/11/20-14:05:04.773284 140257819043392 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033199.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-14:05:04.941411 140257819043392 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033199.sst): second pass time = 0.51's, 255.674'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 250.697'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2103.338'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.05's, 881.262'MB/sec dict compress time = 8247586391.95's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 25595.334'MB/sec zip my value time = 0.51's, unzip length = 0.132'GB zip my value throughput = 255.674'MB/sec zip pipeline throughput = 255.674'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.02 avg-zval = 312.22 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1020.03 avg-zval = 312.22 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 55610 bytes = 187696 } seq expand size = 389270 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 7.0627 value = 3.2574 dict = 1.00 all = 3.1792 } Zip/UnZip{ index = 0.1416 value = 0.3070 dict = 1.00 all = 0.3145 } ---------------------------- total value len = 0.264554 GB avg = 1.022 KB (by entry num) total key len = 0.005177 GB avg = 0.020 KB total ukey len = 0.003106 GB avg = 0.012 KB total ukey num = 0.000258857 Billion total entry num = 0.000258857 Billion write speed all = 0.014164226 MB/sec (with seq num) write speed all = 0.014055480 MB/sec (without seq num) 2024/11/20-14:05:04.995841 140257819043392 [compaction_job.cc:1948] [default] [JOB 8911] Generated table #33199: 129426 keys, 42217207 bytes, temperature: kWarm 2024/11/20-14:05:04.995908 140257819043392 EVENT_LOG_v1 {"time_micros": 1732082704995867, "cf_name": "default", "job": 8911, "event": "table_file_creation", "file_number": 33199, "file_size": 42217207, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 607191178, "table_properties": {"data_size": 40409264, "index_size": 219904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629233, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732082703, "oldest_key_time": 0, "file_creation_time": 1732082703, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 33199, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-14:05:05.068275 140257819043392 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033200.sst) /dev/shm/ToplingTemp/Topling-272819-lAksow 2024/11/20-14:05:05.600093 140257819043392 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033200.sst): first pass time = 0.53's, 252.399'MB/sec 2024/11/20-14:05:06.130611 140257819043392 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033200.sst): old prealloc_size = 147639500, real_size = 40453552 2024/11/20-14:05:06.130816 140257819043392 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033200.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.48 2024/11/20-14:05:06.130927 140257819043392 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033200.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-14:05:06.309839 140257819043392 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033200.sst): second pass time = 0.53's, 248.560'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 233.758'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1534.294'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.245'MB/sec write SST data time = 0.05's, 893.554'MB/sec dict compress time = 8247586391.95's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44656.371'MB/sec zip my value time = 0.53's, unzip length = 0.132'GB zip my value throughput = 248.560'MB/sec zip pipeline throughput = 262.996'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1017.00 avg-zval = 310.48 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1018.26 avg-zval = 310.48 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 23275 bytes = 78576 } seq expand size = 162925 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.6017 value = 3.2755 dict = 1.00 all = 3.2033 } Zip/UnZip{ index = 0.1515 value = 0.3053 dict = 1.00 all = 0.3122 } ---------------------------- total value len = 0.396346 GB avg = 1.021 KB (by entry num) total key len = 0.007766 GB avg = 0.020 KB total ukey len = 0.004659 GB avg = 0.012 KB total ukey num = 0.000388286 Billion total entry num = 0.000388286 Billion write speed all = 0.021219384 MB/sec (with seq num) write speed all = 0.021056277 MB/sec (without seq num) 2024/11/20-14:05:06.359821 140257819043392 [compaction_job.cc:1948] [default] [JOB 8911] Generated table #33200: 129429 keys, 41899638 bytes, temperature: kWarm 2024/11/20-14:05:06.359892 140257819043392 EVENT_LOG_v1 {"time_micros": 1732082706359859, "cf_name": "default", "job": 8911, "event": "table_file_creation", "file_number": 33200, "file_size": 41899638, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 607188172, "table_properties": {"data_size": 40185760, "index_size": 235264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629633, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732082705, "oldest_key_time": 0, "file_creation_time": 1732082705, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 33200, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-14:05:06.400820 140257819043392 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033201.sst) /dev/shm/ToplingTemp/Topling-272819-gaGlit 2024/11/20-14:05:07.031569 140257819043392 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033201.sst): first pass time = 0.63's, 212.806'MB/sec 2024/11/20-14:05:07.681308 140257819043392 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033201.sst): old prealloc_size = 147639500, real_size = 40394080 2024/11/20-14:05:07.681442 140257819043392 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033201.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 310.10 2024/11/20-14:05:07.681504 140257819043392 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033201.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-14:05:07.843624 140257819043392 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033201.sst): second pass time = 0.65's, 202.899'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 257.487'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2100.901'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.280'MB/sec write SST data time = 0.05's, 892.297'MB/sec dict compress time = 8247586391.95's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 24014.971'MB/sec zip my value time = 0.65's, unzip length = 0.132'GB zip my value throughput = 202.899'MB/sec zip pipeline throughput = 202.899'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.99 avg-zval = 310.10 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1017.45 avg-zval = 310.10 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 8465 bytes = 28592 } seq expand size = 59255 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0418 GB } UnZip/Zip{ index = 6.9120 value = 3.2795 dict = 1.00 all = 3.2117 } Zip/UnZip{ index = 0.1447 value = 0.3049 dict = 1.00 all = 0.3114 } ---------------------------- total value len = 0.528035 GB avg = 1.020 KB (by entry num) total key len = 0.010354 GB avg = 0.020 KB total ukey len = 0.006213 GB avg = 0.012 KB total ukey num = 0.000517716 Billion total entry num = 0.000517716 Billion write speed all = 0.028267828 MB/sec (with seq num) write speed all = 0.028050369 MB/sec (without seq num) 2024/11/20-14:05:07.926504 140257819043392 [compaction_job.cc:1948] [default] [JOB 8911] Generated table #33201: 129430 keys, 41790306 bytes, temperature: kWarm 2024/11/20-14:05:07.926573 140257819043392 EVENT_LOG_v1 {"time_micros": 1732082707926541, "cf_name": "default", "job": 8911, "event": "table_file_creation", "file_number": 33201, "file_size": 41790306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 607191648, "table_properties": {"data_size": 40136848, "index_size": 224704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629398, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732082706, "oldest_key_time": 0, "file_creation_time": 1732082706, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 33201, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-14:05:07.972811 140257819043392 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033202.sst) /dev/shm/ToplingTemp/Topling-272819-PR6u2u 2024/11/20-14:05:08.587366 140257819043392 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033202.sst): first pass time = 0.61's, 218.416'MB/sec 2024/11/20-14:05:09.092168 140257819043392 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033202.sst): old prealloc_size = 147639500, real_size = 40415264 2024/11/20-14:05:09.092291 140257819043392 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033202.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.24 2024/11/20-14:05:09.092354 140257819043392 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033202.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-14:05:09.257576 140257819043392 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033202.sst): second pass time = 0.50's, 261.163'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 252.902'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2326.550'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.265'MB/sec write SST data time = 0.04's, 972.329'MB/sec dict compress time = 0.00's, 1271398.524'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35348.329'MB/sec zip my value time = 0.50's, unzip length = 0.132'GB zip my value throughput = 261.163'MB/sec zip pipeline throughput = 261.163'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1017.00 avg-zval = 310.24 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1017.72 avg-zval = 310.24 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13272 bytes = 44800 } seq expand size = 92904 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0418 GB } UnZip/Zip{ index = 6.7996 value = 3.2781 dict = 1.00 all = 3.2088 } Zip/UnZip{ index = 0.1471 value = 0.3051 dict = 1.00 all = 0.3116 } ---------------------------- total value len = 0.659757 GB avg = 1.019 KB (by entry num) total key len = 0.012943 GB avg = 0.020 KB total ukey len = 0.007766 GB avg = 0.012 KB total ukey num = 0.000647145 Billion total entry num = 0.000647145 Billion write speed all = 0.035317128 MB/sec (with seq num) write speed all = 0.035045325 MB/sec (without seq num) 2024/11/20-14:05:09.322266 140257819043392 [compaction_job.cc:1948] [default] [JOB 8911] Generated table #33202: 129429 keys, 41827923 bytes, temperature: kWarm 2024/11/20-14:05:09.322334 140257819043392 EVENT_LOG_v1 {"time_micros": 1732082709322302, "cf_name": "default", "job": 8911, "event": "table_file_creation", "file_number": 33202, "file_size": 41827923, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 607188208, "table_properties": {"data_size": 40154320, "index_size": 228416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629456, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732082707, "oldest_key_time": 0, "file_creation_time": 1732082707, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 33202, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-14:05:09.362360 140257819043392 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033203.sst) /dev/shm/ToplingTemp/Topling-272819-UuWFEt 2024/11/20-14:05:09.897566 140257819043392 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033203.sst): first pass time = 0.54's, 250.804'MB/sec 2024/11/20-14:05:10.888348 140257819043392 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033203.sst): old prealloc_size = 147639500, real_size = 40521536 2024/11/20-14:05:10.888467 140257819043392 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033203.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129423 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.05 avg-zval = 311.15 2024/11/20-14:05:10.888549 140257819043392 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033203.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-14:05:11.118451 140257819043392 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033203.sst): second pass time = 0.99's, 132.952'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 182.554'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2263.893'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.193'MB/sec write SST data time = 0.06's, 673.873'MB/sec dict compress time = 0.04's, 34.405'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35928.437'MB/sec zip my value time = 0.99's, unzip length = 0.132'GB zip my value throughput = 132.952'MB/sec zip pipeline throughput = 248.240'MB/sec entries = 129423 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.05 avg-zval = 311.15 usrkeys = 129423 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1018.81 avg-zval = 311.15 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 32449 bytes = 109536 } seq expand size = 227143 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 7.0728 value = 3.2687 dict = 1.00 all = 3.1958 } Zip/UnZip{ index = 0.1414 value = 0.3059 dict = 1.00 all = 0.3129 } ---------------------------- total value len = 0.791615 GB avg = 1.019 KB (by entry num) total key len = 0.015531 GB avg = 0.020 KB total ukey len = 0.009319 GB avg = 0.012 KB total ukey num = 0.000776568 Billion total entry num = 0.000776568 Billion write speed all = 0.042371463 MB/sec (with seq num) write speed all = 0.042045333 MB/sec (without seq num) 2024/11/20-14:05:11.181360 140257819043392 [compaction_job.cc:1948] [default] [JOB 8911] Generated table #33203: 129423 keys, 41998439 bytes, temperature: kWarm 2024/11/20-14:05:11.181420 140257819043392 EVENT_LOG_v1 {"time_micros": 1732082711181391, "cf_name": "default", "job": 8911, "event": "table_file_creation", "file_number": 33203, "file_size": 41998439, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 607193202, "table_properties": {"data_size": 40269424, "index_size": 219584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588460, "raw_average_key_size": 20, "raw_value_size": 131630120, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129423, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732082709, "oldest_key_time": 0, "file_creation_time": 1732082709, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 33203, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-14:05:11.217434 140257819043392 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033204.sst) /dev/shm/ToplingTemp/Topling-272819-6eKXbu 2024/11/20-14:05:11.230541 140257819043392 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033204.sst): first pass time = 0.01's, 552.277'MB/sec 2024/11/20-14:05:11.285688 140257819043392 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033204.sst): old prealloc_size = 147639500, real_size = 2256648 2024/11/20-14:05:11.285722 140257819043392 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033204.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 6960 prefix = 4 raw-key = 0.0001 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.36 raw-val = 0.0071 GB zip-val = 0.0023 GB avg-val =1017.01 avg-zval = 323.59 2024/11/20-14:05:11.296143 140257819043392 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00/033204.sst): second pass time = 0.05's, 129.540'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.01's, 225.602'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 132.670'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.210'MB/sec write SST data time = 0.00's, 1111.418'MB/sec dict compress time = 0.00's, 74624.266'MB/sec z-dict build time = 0.00's, sample length = 0.076'MB, throughput = 3019.240'MB/sec zip my value time = 0.05's, unzip length = 0.007'GB zip my value throughput = 129.540'MB/sec zip pipeline throughput = 129.540'MB/sec entries = 6960 avg-key = 20.00 avg-zkey = 0.37 avg-val = 1017.01 avg-zval = 323.59 usrkeys = 6960 avg-key = 12.00 avg-zkey = 0.37 avg-val = 1024.01 avg-zval = 323.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 6960 bytes = 22640 } seq expand size = 48720 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0001 GB value = 0.0071 GB dict = 0.08 MB all = 0.0072 GB } __ZipSize{ index = 0.0000 GB value = 0.0023 GB dict = 0.08 MB all = 0.0024 GB } UnZip/Zip{ index = 32.6250 value = 3.1429 dict = 1.00 all = 3.0650 } Zip/UnZip{ index = 0.0307 value = 0.3182 dict = 1.00 all = 0.3263 } ---------------------------- total value len = 0.798742 GB avg = 1.019 KB (by entry num) total key len = 0.015671 GB avg = 0.020 KB total ukey len = 0.009402 GB avg = 0.012 KB total ukey num = 0.000783528 Billion total entry num = 0.000783528 Billion write speed all = 0.042752512 MB/sec (with seq num) write speed all = 0.042423462 MB/sec (without seq num) 2024/11/20-14:05:11.306172 140257819043392 [compaction_job.cc:1948] [default] [JOB 8911] Generated table #33204: 6960 keys, 2354864 bytes, temperature: kWarm 2024/11/20-14:05:11.306221 140257819043392 EVENT_LOG_v1 {"time_micros": 1732082711306196, "cf_name": "default", "job": 8911, "event": "table_file_creation", "file_number": 33204, "file_size": 2354864, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 543253809, "largest_seqno": 605672556, "table_properties": {"data_size": 2252192, "index_size": 2560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 139200, "raw_average_key_size": 20, "raw_value_size": 7078379, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 6960, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1732082711, "oldest_key_time": 0, "file_creation_time": 1732082711, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 33204, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-14:05:11.314558 140257819043392 [compaction_job.cc:703] job-08911: subcompact[0], size: 0.254833 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08911/att-00 [033198,033199,033200,033201,033202,033203,033204] 2024/11/20-14:05:11.362556 140257819043392 (Original Log Time 2024/11/20-14:05:02.046982) EVENT_LOG_v1 {"time_micros": 1732082702046942, "job": 8911, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [33194], "files_L3": [33122, 33123, 33110, 33111, 33112, 33113, 33125], "score": 0.850315, "input_data_size": 286852450, "oldest_snapshot_seqno": 607255784} 2024/11/20-14:05:11.362560 140257819043392 (Original Log Time 2024/11/20-14:05:11.358311) [compaction_job.cc:2022] [default] [JOB 8911] Compacted 1@2 + 7@3 files to L3 => 254832702 bytes 2024/11/20-14:05:11.362561 140257819043392 (Original Log Time 2024/11/20-14:05:11.362478) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 7 0 0 0] max score 0.01, MB/sec: 31.0 rd, 27.5 wr, level 3, files in(1, 7) out(7 +0 blob) MB in(31.2, 242.3 +0.0 blob) out(243.0 +0.0 blob), read-write-amplify(16.5) write-amplify(7.8) OK, records in: 813354, records dropped: 29826 output_compression: Snappy 2024/11/20-14:05:11.362565 140257819043392 (Original Log Time 2024/11/20-14:05:11.362531) EVENT_LOG_v1 {"time_micros": 1732082711362504, "cf": "default", "job": 8911, "event": "compaction_finished", "compaction_time_micros": 9267580, "compaction_time_cpu_micros": 2270463, "output_level": 3, "num_output_files": 7, "total_output_size": 254832702, "num_input_records": 813354, "num_output_records": 783528, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 210845825, "file_range_sync_nanos": 0, "file_fsync_nanos": 6134, "file_prepare_write_nanos": 15022, "lsm_state": [0, 0, 0, 7, 0, 0, 0]} 2024/11/20-14:05:11.483470 140257819043392 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732076400,"starts":1732082701,"executesMs":9451,"compactionJobId":8911,"attempt":0,"compactionInputRawBytes":843456954,"compactionInputZipBytes":286852450,"compactionOutputRawBytes":0,"compactionOutputZipBytes":254832702,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-1430cd7d0cff9bb00dd424051649101c-1136e1125e3f40df-00"}}, response = 2024/11/20-14:05:11.483547 140257819043392 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-08911/att-00: olev 3, work 9.452 s, result 7.662 ms, install 4.273 ms, input{raw 804.383 MiB zip 273.564 MiB}