2024/11/20-17:07:07.482620 140257869399616 DEBG dcompact_worker.cpp:2219: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00: fork to child time = 0.012880 sec 2024/11/20-17:07:07.483169 140257869399616 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00 2024/11/20-17:07:07.489775 140257869399616 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00 2024/11/20-17:07:07.491453 140257869399616 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/20-17:07:07.491666 140257869399616 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/20-17:07:07.542579 140257869399616 [version_set.cc:6211] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-17603/att-00/MANIFEST-019309 2024/11/20-17:07:07.593626 140257869399616 [version_set.cc:6254] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-17603/att-00/MANIFEST-019309 succeeded,manifest_file_number is 19309, next_file_number is 48729, last_sequence is 817548494, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 48492 2024/11/20-17:07:07.593632 140257869399616 [version_set.cc:6269] Column family [default] (ID 0), log number is 0 2024/11/20-17:07:07.593711 140257869399616 [version_set.cc:5724] Creating manifest 48729 2024/11/20-17:07:07.626143 140257869399616 DEBG dcompact_worker.cpp:1225: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/20-17:07:07.654988 140257869399616 [compaction_job.cc:2437] [default] [JOB 17603] Compacting 1@2 + 15@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/20-17:07:07.655000 140257869399616 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 13413 Base level 2, inputs: [48679(27MB)], [48724(38MB) 48725(6666KB) 48668(37MB) 48669(37MB) 48670(38MB) 48671(35MB) 48672(35MB) 48673(37MB) 48674(38MB) 48675(37MB) 48676(37MB) 48587(36MB) 48588(37MB) 48589(36MB) 48705(38MB)] 2024/11/20-17:07:07.661662 140257869399616 [top_zip_table.cc:59] core git_version_hash_info_is:commit 81964c21ebbcc178d8e99dae0fbf61bded2729cc Author: rockeet Date: Mon Nov 11 21:28:46 2024 +0800 Makefile: fix gen git-version-*.cpp GIT_PATH_ARG = :!src/terark/fsa :!tools/fsa :!src/terark/zbs :!tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-17:07:07.661685 140257869399616 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 5dfecb7d107196f044ff5bf78dc39d2e725b9d9b Author: leipeng Date: Thu Nov 7 15:22:45 2024 +0800 Makefile: fix for x86_64 CPU older than haswell GIT_PATH_ARG = src/terark/fsa tools/fsa g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-17:07:07.661704 140257869399616 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit d44dd5ff72707128fe3c916077b43d2093c01f7c Author: leipeng Date: Sun Oct 27 23:38:26 2024 +0800 Fix for msvc, when toplingdb on msvc GIT_PATH_ARG = src/terark/zbs tools/zbs g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/zstd -Iboost-include DEFS = -DTOPLING_IO_WITH_URING=1 -DDIVSUFSORT_API= CXXFLAGS = -fPIC -time -fdiagnostics-color -mcx16 -fno-stack-protector WARNINGS = -Wno-class-memaccess -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -Wno-deprecated-declarations -Wstrict-aliasing=3 RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 LDFLAGS = -fPIC -rdynamic 2024/11/20-17:07:07.661711 140257869399616 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 79614a7e7dc27362fc5151b0797eb7d83c674c7b Author: rockeet Date: Sun Nov 10 15:02:21 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-17:07:07.661720 140257869399616 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit c46b65a52d9c74aa8b87aeb8438620ff2dbc669c Author: rockeet Date: Sun Nov 10 15:02:42 2024 +0800 cpu_has_bmi2.sh: detect by compile c++ code check __BMI2__ g++ (GCC) 12.2.1 20221121 (Red Hat 12.2.1-7) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/20-17:07:08.222605 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048730.sst) /dev/shm/ToplingTemp/Topling-636196-Io8l2n 2024/11/20-17:07:08.733705 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048730.sst): first pass time = 0.51's, 262.627'MB/sec 2024/11/20-17:07:09.920123 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048730.sst): old prealloc_size = 147639500, real_size = 35797632 2024/11/20-17:07:09.920717 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048730.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928666 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0338 GB avg-val = 116.53 avg-zval = 36.41 2024/11/20-17:07:09.921578 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048730.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:10.747626 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048730.sst): second pass time = 1.19's, 91.277'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.364'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3045.742'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 319.406'MB/sec dict compress time = 8225610810.48's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 15784.334'MB/sec zip my value time = 1.19's, unzip length = 0.108'GB zip my value throughput = 91.277'MB/sec zip pipeline throughput = 91.277'MB/sec entries = 928666 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.53 avg-zval = 36.41 usrkeys = 928666 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.53 avg-zval = 36.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928666 bytes = 3366432 } seq expand size = 6500662 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.5819 value = 3.2007 dict = 1.00 all = 3.3537 } Zip/UnZip{ index = 0.0945 value = 0.3124 dict = 1.00 all = 0.2982 } ---------------------------- total value len = 2183.597495 GB avg = 0.300 KB (by entry num) total key len = 191.589295 GB avg = 0.026 KB total ukey len = 134.138905 GB avg = 0.014 KB total ukey num = 9.327060732 Billion total entry num = 7.271210878 Billion write speed all = 134.392016145 MB/sec (with seq num) write speed all = 131.100678575 MB/sec (without seq num) 2024/11/20-17:07:10.796474 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48730: 928666 keys, 40021136 bytes, temperature: kWarm 2024/11/20-17:07:10.796565 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093630796526, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48730, "file_size": 40021136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 438774272, "largest_seqno": 816376477, "table_properties": {"data_size": 33810096, "index_size": 1755200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002648, "raw_average_key_size": 28, "raw_value_size": 108215108, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928666, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093628, "oldest_key_time": 0, "file_creation_time": 1732093628, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48730, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:10.835481 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048731.sst) /dev/shm/ToplingTemp/Topling-636196-0xUkPk 2024/11/20-17:07:11.701398 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048731.sst): first pass time = 0.87's, 155.007'MB/sec 2024/11/20-17:07:12.853354 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048731.sst): old prealloc_size = 147639500, real_size = 35901296 2024/11/20-17:07:12.853939 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048731.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927593 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1082 GB zip-val = 0.0339 GB avg-val = 116.69 avg-zval = 36.51 2024/11/20-17:07:12.854801 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048731.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:13.661734 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048731.sst): second pass time = 1.15's, 94.033'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 49.632'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3189.190'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.342'MB/sec write SST data time = 0.13's, 310.506'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32698.402'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 94.033'MB/sec zip pipeline throughput = 94.033'MB/sec entries = 927593 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.69 avg-zval = 36.51 usrkeys = 927593 avg-key = 20.00 avg-zkey = 1.94 avg-val = 123.39 avg-zval = 36.51 TagRS{ kinds = RS_Key0_TagN bytes = 145040 } TagArray{ size = 887200 bytes = 3216112 } seq expand size = 6210400 multi value expand size = 0 cnt WriteAppend = 34 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.3099 value = 3.1959 dict = 1.00 all = 3.3454 } Zip/UnZip{ index = 0.0970 value = 0.3129 dict = 1.00 all = 0.2989 } ---------------------------- total value len = 2183.711951 GB avg = 0.300 KB (by entry num) total key len = 191.615268 GB avg = 0.026 KB total ukey len = 134.157457 GB avg = 0.014 KB total ukey num = 9.327988325 Billion total entry num = 7.272138471 Billion write speed all = 134.377804916 MB/sec (with seq num) write speed all = 131.086590141 MB/sec (without seq num) 2024/11/20-17:07:13.709743 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48731: 927593 keys, 40120566 bytes, temperature: kWarm 2024/11/20-17:07:13.709810 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093633709779, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48731, "file_size": 40120566, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 816376670, "table_properties": {"data_size": 33869792, "index_size": 1799424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25972604, "raw_average_key_size": 28, "raw_value_size": 108245166, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927593, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093630, "oldest_key_time": 0, "file_creation_time": 1732093630, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48731, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:13.746229 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048732.sst) /dev/shm/ToplingTemp/Topling-636196-6YwIsm 2024/11/20-17:07:14.561778 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048732.sst): first pass time = 0.82's, 164.580'MB/sec 2024/11/20-17:07:15.698895 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048732.sst): old prealloc_size = 147639500, real_size = 35166464 2024/11/20-17:07:15.699491 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048732.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927114 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 116.77 avg-zval = 35.78 2024/11/20-17:07:15.700281 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048732.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:16.532720 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048732.sst): second pass time = 1.14's, 95.279'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.091'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3125.528'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.12's, 315.243'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18187.306'MB/sec zip my value time = 1.14's, unzip length = 0.108'GB zip my value throughput = 95.279'MB/sec zip pipeline throughput = 95.279'MB/sec entries = 927114 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 35.78 usrkeys = 927114 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.77 avg-zval = 35.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927114 bytes = 3244912 } seq expand size = 6489798 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5128 value = 3.2637 dict = 1.00 all = 3.4182 } Zip/UnZip{ index = 0.0951 value = 0.3064 dict = 1.00 all = 0.2926 } ---------------------------- total value len = 2183.826699 GB avg = 0.300 KB (by entry num) total key len = 191.641227 GB avg = 0.026 KB total ukey len = 134.175999 GB avg = 0.014 KB total ukey num = 9.328915439 Billion total entry num = 7.273065585 Billion write speed all = 134.363941846 MB/sec (with seq num) write speed all = 131.072842015 MB/sec (without seq num) 2024/11/20-17:07:16.580436 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48732: 927114 keys, 39265750 bytes, temperature: kWarm 2024/11/20-17:07:16.580496 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093636580467, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48732, "file_size": 39265750, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 635932374, "largest_seqno": 816375782, "table_properties": {"data_size": 33170736, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959192, "raw_average_key_size": 28, "raw_value_size": 108258572, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927114, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093633, "oldest_key_time": 0, "file_creation_time": 1732093633, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48732, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:16.616908 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048733.sst) /dev/shm/ToplingTemp/Topling-636196-Z5gWUk 2024/11/20-17:07:17.421623 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048733.sst): first pass time = 0.80's, 166.795'MB/sec 2024/11/20-17:07:18.613816 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048733.sst): old prealloc_size = 147639500, real_size = 35569520 2024/11/20-17:07:18.614430 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048733.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926256 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 116.90 avg-zval = 36.19 2024/11/20-17:07:18.615277 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048733.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:19.399734 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048733.sst): second pass time = 1.19's, 90.882'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 50.599'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3044.932'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.353'MB/sec write SST data time = 0.13's, 308.629'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34206.737'MB/sec zip my value time = 1.19's, unzip length = 0.108'GB zip my value throughput = 90.882'MB/sec zip pipeline throughput = 90.882'MB/sec entries = 926256 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.90 avg-zval = 36.19 usrkeys = 926256 avg-key = 20.00 avg-zkey = 1.97 avg-val = 123.56 avg-zval = 36.19 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 880715 bytes = 3192608 } seq expand size = 6165005 multi value expand size = 0 cnt WriteAppend = 48 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.1777 value = 3.2306 dict = 1.00 all = 3.3752 } Zip/UnZip{ index = 0.0983 value = 0.3095 dict = 1.00 all = 0.2963 } ---------------------------- total value len = 2183.941147 GB avg = 0.300 KB (by entry num) total key len = 191.667162 GB avg = 0.026 KB total ukey len = 134.194524 GB avg = 0.014 KB total ukey num = 9.329841695 Billion total entry num = 7.273991841 Billion write speed all = 134.350095173 MB/sec (with seq num) write speed all = 131.059109896 MB/sec (without seq num) 2024/11/20-17:07:19.448119 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48733: 926256 keys, 39765334 bytes, temperature: kWarm 2024/11/20-17:07:19.448179 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093639448150, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48733, "file_size": 39765334, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 816376772, "table_properties": {"data_size": 33517600, "index_size": 1820160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935168, "raw_average_key_size": 28, "raw_value_size": 108282572, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926256, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093636, "oldest_key_time": 0, "file_creation_time": 1732093636, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48733, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:19.480553 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048734.sst) /dev/shm/ToplingTemp/Topling-636196-bYMBBk 2024/11/20-17:07:20.233501 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048734.sst): first pass time = 0.75's, 178.263'MB/sec 2024/11/20-17:07:21.520927 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048734.sst): old prealloc_size = 147639500, real_size = 36516176 2024/11/20-17:07:21.521522 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048734.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927036 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.95 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.19 2024/11/20-17:07:21.521652 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048734.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:22.215158 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048734.sst): second pass time = 1.29's, 84.144'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 54.074'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3182.296'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.405'MB/sec write SST data time = 0.12's, 307.776'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 18858.713'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.144'MB/sec zip pipeline throughput = 84.144'MB/sec entries = 927036 avg-key = 28.00 avg-zkey = 1.95 avg-val = 116.78 avg-zval = 37.19 usrkeys = 927036 avg-key = 20.00 avg-zkey = 1.95 avg-val = 116.83 avg-zval = 37.19 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 5910 bytes = 19216 } seq expand size = 41370 multi value expand size = 0 cnt WriteAppend = 64 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.2392 value = 3.1404 dict = 1.00 all = 3.5754 } Zip/UnZip{ index = 0.0977 value = 0.3184 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 2184.049449 GB avg = 0.300 KB (by entry num) total key len = 191.693119 GB avg = 0.026 KB total ukey len = 134.213065 GB avg = 0.014 KB total ukey num = 9.330768731 Billion total entry num = 7.274918877 Billion write speed all = 134.336298574 MB/sec (with seq num) write speed all = 131.045417862 MB/sec (without seq num) 2024/11/20-17:07:22.262204 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48734: 927036 keys, 37538769 bytes, temperature: kWarm 2024/11/20-17:07:22.262267 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093642262237, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48734, "file_size": 37538769, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 538604047, "table_properties": {"data_size": 34473472, "index_size": 1810752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957008, "raw_average_key_size": 28, "raw_value_size": 108260768, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927036, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093639, "oldest_key_time": 0, "file_creation_time": 1732093639, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48734, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:22.298468 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048735.sst) /dev/shm/ToplingTemp/Topling-636196-q8iFsn 2024/11/20-17:07:23.052398 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048735.sst): first pass time = 0.75's, 178.032'MB/sec 2024/11/20-17:07:24.350039 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048735.sst): old prealloc_size = 147639500, real_size = 36417152 2024/11/20-17:07:24.350660 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048735.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926887 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.95 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.80 avg-zval = 37.09 2024/11/20-17:07:24.350800 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048735.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:25.041634 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048735.sst): second pass time = 1.30's, 83.481'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 54.113'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3156.126'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.406'MB/sec write SST data time = 0.12's, 309.425'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35534.419'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.481'MB/sec zip pipeline throughput = 83.481'MB/sec entries = 926887 avg-key = 28.00 avg-zkey = 1.95 avg-val = 116.80 avg-zval = 37.09 usrkeys = 926887 avg-key = 20.00 avg-zkey = 1.95 avg-val = 116.81 avg-zval = 37.09 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 936 bytes = 3056 } seq expand size = 6552 multi value expand size = 0 cnt WriteAppend = 35 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.2405 value = 3.1495 dict = 1.00 all = 3.5865 } Zip/UnZip{ index = 0.0977 value = 0.3175 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 2184.157721 GB avg = 0.300 KB (by entry num) total key len = 191.719072 GB avg = 0.026 KB total ukey len = 134.231603 GB avg = 0.014 KB total ukey num = 9.331695618 Billion total entry num = 7.275845764 Billion write speed all = 134.322420476 MB/sec (with seq num) write speed all = 131.031646419 MB/sec (without seq num) 2024/11/20-17:07:25.089005 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48735: 926887 keys, 37423415 bytes, temperature: kWarm 2024/11/20-17:07:25.089081 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093645089036, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48735, "file_size": 37423415, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 538604114, "table_properties": {"data_size": 34375024, "index_size": 1810240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952836, "raw_average_key_size": 28, "raw_value_size": 108264966, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926887, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093642, "oldest_key_time": 0, "file_creation_time": 1732093642, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48735, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:25.124350 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048736.sst) /dev/shm/ToplingTemp/Topling-636196-b8r41m 2024/11/20-17:07:25.901873 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048736.sst): first pass time = 0.78's, 172.629'MB/sec 2024/11/20-17:07:27.225341 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048736.sst): old prealloc_size = 147639500, real_size = 36531824 2024/11/20-17:07:27.225926 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048736.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926768 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.82 avg-zval = 37.20 2024/11/20-17:07:27.226403 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048736.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:27.943286 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048736.sst): second pass time = 1.32's, 81.858'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 54.466'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3215.825'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.397'MB/sec write SST data time = 0.13's, 292.045'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 18127.970'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.858'MB/sec zip pipeline throughput = 81.858'MB/sec entries = 926768 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.82 avg-zval = 37.20 usrkeys = 926768 avg-key = 20.00 avg-zkey = 1.97 avg-val = 120.33 avg-zval = 37.20 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 464596 bytes = 1568032 } seq expand size = 3252172 multi value expand size = 0 cnt WriteAppend = 48 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.1766 value = 3.1402 dict = 1.00 all = 3.4324 } Zip/UnZip{ index = 0.0983 value = 0.3185 dict = 1.00 all = 0.2913 } ---------------------------- total value len = 2184.269241 GB avg = 0.300 KB (by entry num) total key len = 191.745021 GB avg = 0.026 KB total ukey len = 134.250138 GB avg = 0.014 KB total ukey num = 9.332622386 Billion total entry num = 7.276772532 Billion write speed all = 134.308159472 MB/sec (with seq num) write speed all = 131.017506074 MB/sec (without seq num) 2024/11/20-17:07:27.991691 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48736: 926768 keys, 39102889 bytes, temperature: kWarm 2024/11/20-17:07:27.991741 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093647991713, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48736, "file_size": 39102889, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 541395865, "table_properties": {"data_size": 34478560, "index_size": 1821376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949504, "raw_average_key_size": 28, "raw_value_size": 108268260, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926768, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093645, "oldest_key_time": 0, "file_creation_time": 1732093645, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48736, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:28.038132 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048737.sst) /dev/shm/ToplingTemp/Topling-636196-KPYCbm 2024/11/20-17:07:28.887286 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048737.sst): first pass time = 0.85's, 158.066'MB/sec 2024/11/20-17:07:30.148057 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048737.sst): old prealloc_size = 147639500, real_size = 36072064 2024/11/20-17:07:30.148590 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048737.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926187 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0341 GB avg-val = 116.91 avg-zval = 36.79 2024/11/20-17:07:30.149334 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048737.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:30.985882 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048737.sst): second pass time = 1.26's, 85.936'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.435'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3473.707'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 306.043'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 21749.235'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.936'MB/sec zip pipeline throughput = 85.936'MB/sec entries = 926187 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 36.79 usrkeys = 926187 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.47 avg-zval = 36.79 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 735332 bytes = 2665600 } seq expand size = 5147324 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5134 value = 3.1775 dict = 1.00 all = 3.3773 } Zip/UnZip{ index = 0.0951 value = 0.3147 dict = 1.00 all = 0.2961 } ---------------------------- total value len = 2184.382673 GB avg = 0.300 KB (by entry num) total key len = 191.770955 GB avg = 0.026 KB total ukey len = 134.268662 GB avg = 0.014 KB total ukey num = 9.333548573 Billion total entry num = 7.277698719 Billion write speed all = 134.292940824 MB/sec (with seq num) write speed all = 131.002434513 MB/sec (without seq num) 2024/11/20-17:07:31.039110 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48737: 926187 keys, 39740934 bytes, temperature: kWarm 2024/11/20-17:07:31.039201 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093651039155, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48737, "file_size": 39740934, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 816374487, "table_properties": {"data_size": 34078384, "index_size": 1761920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933236, "raw_average_key_size": 28, "raw_value_size": 108284626, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926187, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093648, "oldest_key_time": 0, "file_creation_time": 1732093648, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48737, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:31.103452 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048738.sst) /dev/shm/ToplingTemp/Topling-636196-qIi6tn 2024/11/20-17:07:32.123651 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048738.sst): first pass time = 1.02's, 131.567'MB/sec 2024/11/20-17:07:33.465722 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048738.sst): old prealloc_size = 147639500, real_size = 35610752 2024/11/20-17:07:33.466278 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048738.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924998 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0336 GB avg-val = 117.10 avg-zval = 36.35 2024/11/20-17:07:33.467141 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048738.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:34.338925 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048738.sst): second pass time = 1.34's, 80.753'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 45.338'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3267.415'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 311.366'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31246.849'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.753'MB/sec zip pipeline throughput = 130.874'MB/sec entries = 924998 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 36.35 usrkeys = 924998 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 36.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924998 bytes = 3121888 } seq expand size = 6474986 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5121 value = 3.2219 dict = 1.00 all = 3.3904 } Zip/UnZip{ index = 0.0951 value = 0.3104 dict = 1.00 all = 0.2950 } ---------------------------- total value len = 2184.497466 GB avg = 0.300 KB (by entry num) total key len = 191.796855 GB avg = 0.026 KB total ukey len = 134.287162 GB avg = 0.014 KB total ukey num = 9.334473571 Billion total entry num = 7.278623717 Billion write speed all = 134.275446460 MB/sec (with seq num) write speed all = 130.985145454 MB/sec (without seq num) 2024/11/20-17:07:34.386650 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48738: 924998 keys, 39587982 bytes, temperature: kWarm 2024/11/20-17:07:34.386718 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093654386687, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48738, "file_size": 39587982, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 717440672, "largest_seqno": 816376266, "table_properties": {"data_size": 33619440, "index_size": 1759872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899944, "raw_average_key_size": 28, "raw_value_size": 108317796, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924998, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093651, "oldest_key_time": 0, "file_creation_time": 1732093651, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48738, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:34.425311 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048739.sst) /dev/shm/ToplingTemp/Topling-636196-8dmlom 2024/11/20-17:07:35.294417 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048739.sst): first pass time = 0.87's, 154.439'MB/sec 2024/11/20-17:07:36.617537 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048739.sst): old prealloc_size = 147639500, real_size = 35934464 2024/11/20-17:07:36.618137 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048739.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929483 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0339 GB avg-val = 116.40 avg-zval = 36.48 2024/11/20-17:07:36.619035 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048739.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:37.538915 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048739.sst): second pass time = 1.32's, 81.812'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 43.725'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3050.552'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 303.655'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36361.174'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.812'MB/sec zip pipeline throughput = 165.671'MB/sec entries = 929483 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.40 avg-zval = 36.48 usrkeys = 929483 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.36 avg-zval = 36.48 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 924190 bytes = 3350208 } seq expand size = 6469330 multi value expand size = 0 cnt WriteAppend = 22 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0403 GB } UnZip/Zip{ index = 10.3781 value = 3.1905 dict = 1.00 all = 3.3316 } Zip/UnZip{ index = 0.0964 value = 0.3134 dict = 1.00 all = 0.3002 } ---------------------------- total value len = 2184.612127 GB avg = 0.300 KB (by entry num) total key len = 191.822880 GB avg = 0.026 KB total ukey len = 134.305751 GB avg = 0.014 KB total ukey num = 9.335403054 Billion total entry num = 7.279553200 Billion write speed all = 134.259119520 MB/sec (with seq num) write speed all = 130.968993259 MB/sec (without seq num) 2024/11/20-17:07:37.587286 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48739: 929483 keys, 40286356 bytes, temperature: kWarm 2024/11/20-17:07:37.587355 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093657587324, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48739, "file_size": 40286356, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 816375889, "table_properties": {"data_size": 33910704, "index_size": 1791232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26025524, "raw_average_key_size": 28, "raw_value_size": 108192214, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929483, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093654, "oldest_key_time": 0, "file_creation_time": 1732093654, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48739, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:37.635091 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048740.sst) /dev/shm/ToplingTemp/Topling-636196-RVP4to 2024/11/20-17:07:38.434315 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048740.sst): first pass time = 0.80's, 167.943'MB/sec 2024/11/20-17:07:39.751706 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048740.sst): old prealloc_size = 147639500, real_size = 35923808 2024/11/20-17:07:39.752267 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048740.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923261 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0339 GB avg-val = 117.37 avg-zval = 36.74 2024/11/20-17:07:39.752871 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048740.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:40.591118 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048740.sst): second pass time = 1.32's, 82.306'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.552'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3295.889'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 306.866'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 20766.130'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.306'MB/sec zip pipeline throughput = 82.306'MB/sec entries = 923261 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.37 avg-zval = 36.74 usrkeys = 923261 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.02 avg-zval = 36.74 TagRS{ kinds = RS_Key0_TagN bytes = 144360 } TagArray{ size = 613271 bytes = 2146464 } seq expand size = 4292897 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.4422 value = 3.1943 dict = 1.00 all = 3.4348 } Zip/UnZip{ index = 0.0958 value = 0.3131 dict = 1.00 all = 0.2911 } ---------------------------- total value len = 2184.724787 GB avg = 0.300 KB (by entry num) total key len = 191.848731 GB avg = 0.026 KB total ukey len = 134.324217 GB avg = 0.014 KB total ukey num = 9.336326315 Billion total entry num = 7.280476461 Billion write speed all = 134.243796100 MB/sec (with seq num) write speed all = 130.953819871 MB/sec (without seq num) 2024/11/20-17:07:40.638486 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48740: 923261 keys, 39076376 bytes, temperature: kWarm 2024/11/20-17:07:40.638540 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093660638512, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48740, "file_size": 39076376, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 659025249, "table_properties": {"data_size": 33924496, "index_size": 1768320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25851308, "raw_average_key_size": 28, "raw_value_size": 108366446, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923261, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093657, "oldest_key_time": 0, "file_creation_time": 1732093657, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48740, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:40.679596 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048741.sst) /dev/shm/ToplingTemp/Topling-636196-AXxdho 2024/11/20-17:07:41.487265 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048741.sst): first pass time = 0.81's, 166.185'MB/sec 2024/11/20-17:07:42.605033 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048741.sst): old prealloc_size = 147639500, real_size = 33686128 2024/11/20-17:07:42.605584 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048741.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923570 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.32 avg-zval = 34.32 2024/11/20-17:07:42.606431 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048741.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:43.460256 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048741.sst): second pass time = 1.12's, 97.010'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 44.308'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3375.657'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 297.554'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32388.409'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 97.010'MB/sec zip pipeline throughput = 97.010'MB/sec entries = 923570 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.32 avg-zval = 34.32 usrkeys = 923570 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.32 avg-zval = 34.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923570 bytes = 3347952 } seq expand size = 6464990 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5146 value = 3.4184 dict = 1.00 all = 3.5421 } Zip/UnZip{ index = 0.0951 value = 0.2925 dict = 1.00 all = 0.2823 } ---------------------------- total value len = 2184.839609 GB avg = 0.300 KB (by entry num) total key len = 191.874591 GB avg = 0.026 KB total ukey len = 134.342688 GB avg = 0.014 KB total ukey num = 9.337249885 Billion total entry num = 7.281400031 Billion write speed all = 134.229988580 MB/sec (with seq num) write speed all = 130.940128175 MB/sec (without seq num) 2024/11/20-17:07:43.507437 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48741: 923570 keys, 37892076 bytes, temperature: kWarm 2024/11/20-17:07:43.507499 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093663507470, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48741, "file_size": 37892076, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 542625158, "largest_seqno": 816376047, "table_properties": {"data_size": 31698336, "index_size": 1756736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859960, "raw_average_key_size": 28, "raw_value_size": 108357832, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923570, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093660, "oldest_key_time": 0, "file_creation_time": 1732093660, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48741, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:43.546452 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048742.sst) /dev/shm/ToplingTemp/Topling-636196-pzqsll 2024/11/20-17:07:44.429123 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048742.sst): first pass time = 0.88's, 152.064'MB/sec 2024/11/20-17:07:45.549621 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048742.sst): old prealloc_size = 147639500, real_size = 33978096 2024/11/20-17:07:45.550230 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048742.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924902 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0320 GB avg-val = 117.12 avg-zval = 34.58 2024/11/20-17:07:45.551075 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048742.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:46.395528 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048742.sst): second pass time = 1.12's, 96.741'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.007'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3005.978'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 303.447'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31258.752'MB/sec zip my value time = 1.12's, unzip length = 0.108'GB zip my value throughput = 96.741'MB/sec zip pipeline throughput = 96.741'MB/sec entries = 924902 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 34.58 usrkeys = 924902 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.12 avg-zval = 34.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924902 bytes = 3237168 } seq expand size = 6474314 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5129 value = 3.3864 dict = 1.00 all = 3.5254 } Zip/UnZip{ index = 0.0951 value = 0.2953 dict = 1.00 all = 0.2837 } ---------------------------- total value len = 2184.954404 GB avg = 0.300 KB (by entry num) total key len = 191.900489 GB avg = 0.026 KB total ukey len = 134.361186 GB avg = 0.014 KB total ukey num = 9.338174787 Billion total entry num = 7.282324933 Billion write speed all = 134.215684765 MB/sec (with seq num) write speed all = 130.925951831 MB/sec (without seq num) 2024/11/20-17:07:46.440611 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48742: 924902 keys, 38071174 bytes, temperature: kWarm 2024/11/20-17:07:46.440677 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093666440646, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48742, "file_size": 38071174, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 664578877, "largest_seqno": 816376798, "table_properties": {"data_size": 31987104, "index_size": 1759552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897256, "raw_average_key_size": 28, "raw_value_size": 108320568, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924902, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093663, "oldest_key_time": 0, "file_creation_time": 1732093663, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48742, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:46.474497 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048743.sst) /dev/shm/ToplingTemp/Topling-636196-JNCOSj 2024/11/20-17:07:47.293571 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048743.sst): first pass time = 0.82's, 163.872'MB/sec 2024/11/20-17:07:48.443773 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048743.sst): old prealloc_size = 147639500, real_size = 35363552 2024/11/20-17:07:48.444363 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048743.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929299 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0334 GB avg-val = 116.43 avg-zval = 35.90 2024/11/20-17:07:48.445193 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048743.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/20-17:07:49.272948 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048743.sst): second pass time = 1.15's, 94.140'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.601'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3126.563'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 312.870'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 21571.765'MB/sec zip my value time = 1.15's, unzip length = 0.108'GB zip my value throughput = 94.140'MB/sec zip pipeline throughput = 94.140'MB/sec entries = 929299 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.43 avg-zval = 35.90 usrkeys = 929299 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.43 avg-zval = 35.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929299 bytes = 3252560 } seq expand size = 6505093 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5109 value = 3.2431 dict = 1.00 all = 3.4006 } Zip/UnZip{ index = 0.0951 value = 0.3084 dict = 1.00 all = 0.2941 } ---------------------------- total value len = 2185.069107 GB avg = 0.300 KB (by entry num) total key len = 191.926509 GB avg = 0.026 KB total ukey len = 134.379772 GB avg = 0.014 KB total ukey num = 9.339104086 Billion total entry num = 7.283254232 Billion write speed all = 134.201825761 MB/sec (with seq num) write speed all = 130.912207525 MB/sec (without seq num) 2024/11/20-17:07:49.320087 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48743: 929299 keys, 39468992 bytes, temperature: kWarm 2024/11/20-17:07:49.320152 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093669320122, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48743, "file_size": 39468992, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 664578947, "largest_seqno": 816375629, "table_properties": {"data_size": 33362768, "index_size": 1768256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26020372, "raw_average_key_size": 28, "raw_value_size": 108197470, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929299, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093666, "oldest_key_time": 0, "file_creation_time": 1732093666, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48743, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:49.356017 140257869399616 [top_zip_table_builder.cc:960] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048744.sst) /dev/shm/ToplingTemp/Topling-636196-XLfgjo 2024/11/20-17:07:49.522905 140257869399616 [top_zip_table_builder.cc:1378] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048744.sst): first pass time = 0.17's, 264.750'MB/sec 2024/11/20-17:07:49.883395 140257869399616 [top_zip_table_builder.cc:2798] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048744.sst): old prealloc_size = 147639500, real_size = 12709632 2024/11/20-17:07:49.883661 140257869399616 [top_zip_table_builder.cc:1799] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048744.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 304957 prefix = 4 raw-key = 0.0049 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.97 raw-val = 0.0356 GB zip-val = 0.0120 GB avg-val = 116.84 avg-zval = 39.46 2024/11/20-17:07:49.883996 140257869399616 [top_zip_table_builder.cc:1325] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048744.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.018 GB, reorder workingMem = 0.0007 GB, waited 0.000 sec, Key+Value = 0.044 GB 2024/11/20-17:07:50.136336 140257869399616 [top_zip_table_builder.cc:3129] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00/048744.sst): second pass time = 0.36's, 99.071'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 55.767'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2692.604'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.366'MB/sec write SST data time = 0.04's, 317.976'MB/sec dict compress time = 140278.07's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.363'MB, throughput = 12331.670'MB/sec zip my value time = 0.36's, unzip length = 0.036'GB zip my value throughput = 99.071'MB/sec zip pipeline throughput = 99.071'MB/sec entries = 304957 avg-key = 28.00 avg-zkey = 1.97 avg-val = 116.84 avg-zval = 39.46 usrkeys = 304957 avg-key = 20.00 avg-zkey = 1.97 avg-val = 123.84 avg-zval = 39.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 304957 bytes = 1105488 } seq expand size = 2134699 multi value expand size = 0 cnt WriteAppend = 26 UnZipSize{ index = 0.0061 GB value = 0.0356 GB dict = 0.36 MB all = 0.0442 GB } __ZipSize{ index = 0.0006 GB value = 0.0120 GB dict = 0.36 MB all = 0.0141 GB } UnZip/Zip{ index = 10.1576 value = 2.9612 dict = 1.00 all = 3.1313 } Zip/UnZip{ index = 0.0984 value = 0.3377 dict = 1.00 all = 0.3194 } ---------------------------- total value len = 2185.106873 GB avg = 0.300 KB (by entry num) total key len = 191.935048 GB avg = 0.026 KB total ukey len = 134.385871 GB avg = 0.014 KB total ukey num = 9.339409043 Billion total entry num = 7.283559189 Billion write speed all = 134.197898508 MB/sec (with seq num) write speed all = 130.908302887 MB/sec (without seq num) 2024/11/20-17:07:50.157776 140257869399616 [compaction_job.cc:1948] [default] [JOB 17603] Generated table #48744: 304957 keys, 14106043 bytes, temperature: kWarm 2024/11/20-17:07:50.157831 140257869399616 EVENT_LOG_v1 {"time_micros": 1732093670157803, "cf_name": "default", "job": 17603, "event": "table_file_creation", "file_number": 48744, "file_size": 14106043, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 505009814, "largest_seqno": 816374950, "table_properties": {"data_size": 12032752, "index_size": 600448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 8538796, "raw_average_key_size": 28, "raw_value_size": 35631558, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 304957, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732093669, "oldest_key_time": 0, "file_creation_time": 1732093669, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "92E45FH6K6HCJOD41QCQ", "orig_file_number": 48744, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/20-17:07:50.172376 140257869399616 [compaction_job.cc:703] job-17603: subcompact[0], size: 0.561468 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17603/att-00 [048730,048731,048732,048733,048734,048735,048736,048737,048738,048739,048740,048741,048742,048743,048744] 2024/11/20-17:07:50.264249 140257869399616 (Original Log Time 2024/11/20-17:07:07.655042) EVENT_LOG_v1 {"time_micros": 1732093627655009, "job": 17603, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [48679], "files_L3": [48724, 48725, 48668, 48669, 48670, 48671, 48672, 48673, 48674, 48675, 48676, 48587, 48588, 48589, 48705], "score": 0.98961, "input_data_size": 584259356, "oldest_snapshot_seqno": 817548424} 2024/11/20-17:07:50.264253 140257869399616 (Original Log Time 2024/11/20-17:07:50.262848) [compaction_job.cc:2022] [default] [JOB 17603] Compacted 1@2 + 15@3 files to L3 => 561467792 bytes 2024/11/20-17:07:50.264254 140257869399616 (Original Log Time 2024/11/20-17:07:50.264178) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 13.7 rd, 13.2 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 530.1 +0.0 blob) out(535.5 +0.0 blob), read-write-amplify(40.3) write-amplify(19.8) OK, records in: 13360648, records dropped: 83671 output_compression: Snappy 2024/11/20-17:07:50.264258 140257869399616 (Original Log Time 2024/11/20-17:07:50.264228) EVENT_LOG_v1 {"time_micros": 1732093670264202, "cf": "default", "job": 17603, "event": "compaction_finished", "compaction_time_micros": 42517324, "compaction_time_cpu_micros": 19005497, "output_level": 3, "num_output_files": 15, "total_output_size": 561467792, "num_input_records": 13360648, "num_output_records": 13276977, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 372859522, "file_range_sync_nanos": 0, "file_fsync_nanos": 12481, "file_prepare_write_nanos": 47660, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/20-17:07:50.388945 140257869399616 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732076400,"starts":1732093627,"executesMs":42772,"compactionJobId":17603,"attempt":0,"compactionInputRawBytes":1935100560,"compactionInputZipBytes":584259356,"compactionOutputRawBytes":0,"compactionOutputZipBytes":561467792,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-575d1c2d9e93172ae3cd92715c54e70b-326940bf22419168-00"}}, response = 2024/11/20-17:07:50.389038 140257869399616 INFO dcompact_worker.cpp:1464: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-20T12.20.00/.rocksdb/job-17603/att-00: olev 3, work 42.772 s, result 7.782 ms, install 1.426 ms, input{raw 1.802 GiB zip 557.193 MiB}